Ball retrieval system and method

ABSTRACT

A tennis ball retriever for retrieving a tennis ball on and about a tennis court and for controllably releasing the ball includes a front-end ball collector having two spaced-apart blades defining a ball collection space. The retriever includes a means for gripping and securing a ball positioned at the fore end, and one or more detents for retaining the collected ball in the collection space until a programmed or manually-directed release toward a target location such as a tennis baseline. The retriever includes a processor-controller for controlling and directing the retriever during a retrieval session. The retriever may be remote-controlled such as by voice commands.

CROSS REFERENCE TO RELATED APPLICATIONS

The present application is a continuation-in-part application Ser. No.16/866,480 filed on May 4, 2020, and entitled “Ball Retriever System andMethod”, which was a continuation-in-part (“CIP Bypass”) application ofPCT Application No. PCT/US19/60143, filed on 6 Nov. 2019 and claimingpriority to U.S. Provisional Patent Application No. 62/756,450, filed onNov. 6, 2018 and entitled “Tennis Ball Retriever System and Method”, andto U.S. Provisional Patent Application No. 62/908,267, filed on Sep. 30,2019 and entitled “Tennis Ball Retrieval System and Method”, all ofwhich are herein incorporated by reference.

FIELD OF THE INVENTION

The invention relates to a method and system for retrieving andreturning a ball on a sports playing surface during play, and moreparticularly to a method and system for locating, retrieving, andreturning a tennis ball to a programmed, manually-directed, or signaledtarget location on a tennis court during match play.

BACKGROUND OF THE INVENTION

FIG. 1 shows the layout and the dimensions of a tennis court, where eachbaseline is the line intersected by its “center mark” and behind which aplayer is positioned when serving. The dimensions are as follows:

-   Baseline: Doubles 36 feet (11 m)/Singles 27 feet (8.23 m) width-   Center mark: 4 inches (10.16 cm) length-   Service line: 27 feet (8.23 m) width (same as singles baseline)-   Center service line on each side: 21 feet (6.4 m) in length (from    net)-   Baseline: 39 feet (11.89 m) in length (from net, same for both    Doubles and Singles court size)-   The serving player is therefore at least 39 feet (11.89 m) from the    net.

A pickle ball court is also rectangular in shape, divided into two sidesby a net, but is smaller than a tennis court. A platform tennis court is4 feet longer compared to a pickleball court but just as wide. Platformtennis has a net that is two feet higher and allows players to comeright up to the net. Pickleball has a shorter net and does not allowplayers to come right up to the net—this is called the no volley zone.The courts' surfaces resemble that of a tennis hard court, although thelatter can vary since tennis is played not just on hard courts but alsoon clay/composites, grass, and differing indoor versus outdoorhard-court surfaces.

Play conditions can vary also according to the surface type and whenoutdoors with environmental conditions including wind and temperature.An element common to all these play conditions, however, is that theamount of time spent in actual play (versus “non-play”, i.e. planned orforced pauses) is adversely impacted by players having to retrieve looseballs both while serving and after completing points, slowing the paceand thus affecting the quality and efficiency of play.

Professional tournaments address the issue by employing ball-persons forretrieving loose tennis balls to expedite play, a luxury not availablefor most due to the lack of fiscal or available manpower resources.Court time is typically expensive, and of course time is money. Playerstherefore often opt to play on despite the presence of loose balls onthe court, which pose both a distraction and a safety hazard (a playermay step on a ball and incur injury).

Although there have been solutions proposed or marketed to assistplayers in retrieving and picking up loose tennis balls, these are notdesigned for in-play. Most of these tennis ball collection/retrievalsystems are just manual collection devices requiring walking to eachball and therefore do not promote increased play time.

U.S. Pat. No. 10,011,208, “Dual Function Robot and Storage Bin”, H.Eletrabi, (“Eletrabi”), issued Jul. 3, 2018 and incorporated herein byreference, describes an automated robotic rollable object (tennis ball)collector system; its design is primarily directed to the collection oftennis balls on the court, but only “en masse” after a practice sessionor the like, which necessitates a cessation in play on the court withoutpromoting continuing play as during a match. Eletrabi also disclosesthat its size and weight make power consumption a concern.

U.S. Patent Application Publication No. 2020/0086182, “Autonomous TennisBall Retrieval Robot”, New et al., published Mar. 19, 2020 andincorporated herein by reference, describes an autonomous robot thatcollects tennis balls on a tennis court surface and transfers the ballsinto a launcher (or “shooter”). The robot is designed solely as a ballretriever/shooter for a practice session on one court side opposite thatof the player and is not configured for match play conditionsnecessitating retrievals on both court sides and off-court and return toa player during the course of a match.

Another automated approach, “Ballbot: A Low-Cost Robot for Tennis BallRetrieval”, John Wang, Technical Report No. UCB/EECS-2012-157(hereinafter “Wang”),http://www.eecs.berkeley.edu/Pubs/TechRpts/2012/EECS-2012-157.html, Jun.2, 2012, describes a mobile robot platform; as with Eletrabi, all thesystem components are on-board, including processing, computer vision,and wireless communication. Wang's computer vision system employson-board cameras with a framerate of 100 frames/s at a 320×240resolution interfaced with a PandaBoard via USB. The motion planning andcontrol program calculates retrieval paths and commands the robotsteering along the mapped route, optimizing the route by length andother factors. The “Path Planner” uses a lattice based planning approachthat encodes differential constraints into the route without the needfor post processing, factoring in the net as a fixed obstacle while alsoaccounting for other static obstacles such as benches and dynamicobstacles such as players. Wang notes that this adds to edge costs butalso observes that a sparsity of obstacles allows for processingoptimization and increased system response time and retrieval speed. Theball location/search algorithm is a heuristic based search model, usinga version of Moving-Target (MT) search called Lazy MT-Adaptive A*,described in S. Koenig, M. Likhachev, and X. Sun, “Speeding Up MovingTarget Search,” Proceedings of the 6th International Joint Conference onAutonomous Agents and Multiagent Systems, ser. AAMAS '07. New York,N.Y., USA: ACM, 2007, pp. 188:1-188:8. [Online]. Available:http://doi.acm.org/10.1145/1329125.1329353. The algorithm works well forthe situation where both the agent (robot) and the goal (ball) can move.MT-Adaptive A* incorporates two key concepts compared to A*, (i)Heuristic update after path computation, and (ii) Heuristic correctionfor Moving Target.

Wang uses a closed loop controller that enables it to use localizationinformation to follow planned paths accurately. The controller has twocomponents—speed control, whereby the controller commands speeds to theArduino, which then runs PID control based on a wheel encoder tomaintain the commanded speed and steering control; and, steering controlbase as described in “Stanley: the robot that won the DARPA GrandChallenge,” Springer Tracts in Advanced Robotics, M. Buehler, K.Iagnemma, and S. Singh, Eds. Springer Berlin/Heidelberg, 2007, vol. 36,pp. 1-43, 10.1007/978-3-54073429-1 1. [Online]. Available:http://dx.doi.org/10.1007/978-3-540-73429-1 1. The steering controlleris based on a nonlinear feedback function of the cross-track error x(t)which measures the lateral distance of the center of the robot's frontwheels from the nearest point on the trajectory.

Like Eletrabi, the Wang robotic device is impractical for match play dueto its large size, weight, processor overhead, and consequent powerdemand. As with other prior art approaches, it employs a ball pickupmechanism, adding to its weight, complexity, and power overhead thataltogether limit its collection capability. The ball pickup mechanismincludes two motors, with gearboxes, and draw 6 A, providing 130 oz-inof torque necessary to compress the tennis ball as it enters the pickup.

It is therefore desirable to provide a reliable and energy-efficientapparatus, system, and method for retrieving and returning a tennis ballto a player during play.

BRIEF SUMMARY OF THE INVENTION

According to the invention, a tennis ball retriever for retrieving atennis ball on and about a tennis court and for controllably releasingthe ball includes a front-end ball collector having two spaced-apartblades defining a ball collection space. The retriever includes a meansfor gripping and securing a ball positioned at the fore end, and one ormore detents for retaining the collected ball in the collection spaceuntil a programmed or manually-directed release toward a target locationsuch as a tennis baseline. The retriever includes a processor-controllerfor controlling and directing the retriever during a retrieval session.The retriever may be remote-controlled such as by voice commands.

Also, according to the invention, a remote-controlled retriever forcollecting a ball on a sports playing surface and for controllablyreleasing the ball includes a front-end ball collector having twospaced-apart blades defining a ball collection space. Each bladeincludes a passive or an active detent for detaining the collected ballin the collection space until a programmed or manually-directed groundrelease utilizing the retriever forward movement toward a targetlocation such as a tennis baseline The retriever optionally includes ameans for ejecting the ball from the ball collection space to augment aground release. The retriever includes a processor-controller forcontrolling and directing the retriever during a retrieval session. Theretriever can be configured for a tennis court for tennis ball retrievalas well as for other court sports including paddle tennis, platformtennis, and pickleball.

In one embodiment, the passive detent is a concave portion on thesidewall of each blade to ensure retaining a collected ball until itsplanned release. A lower horizontally-extending ball guide cooperateswith the opposing blade counterpart structure to guide the ball onrelease toward the target (and prevent the ball from reentering thedetent area during the release). A ball proximity sensor signals thepresence or absence of a ball in the collection space. Active detents beemployed. A ball ejector can be added to expand the functionality of theretriever.

The retriever includes means for determining its location and isinitially positioned in its standby location (e.g. in a dockingstation), preferably by a net post. Upon powerup/startup, aninitialization routine establishes its standby location relative to aprogrammed-in court grid, with a processor programmed to return theretriever to standby following retrievals. The retriever can include anorientation sensor that upon powerup and initialization establishes aprecise orientation relative to the court and the court grid. Processorprogramming can direct the retriever to commence parallel to and nearthe net on the directed court side, with the remote control redirectingthe path (manually or in some designs via voice commands, either via theremote or without a remote directly to an acoustic pickup on theretriever or docking station). Additional proximity sensors on theretriever can sense and maintain a desired clearance from the net orother objects.

The docking station embodiment preferably includes means for attachingand securing to a net post and charging means for charging theretriever's power supply between retrievals. The docking station mayalso include means (e.g. a transponder or a transceiver) to communicateor ping its location to the retriever and guide it back directly after aretrieval.

In another embodiment, the invention includes a portable wheeled frameadapted to hold the retriever-in-docking-station with means forattaching the assembly to a net post. The frame can include atelescoping handle that when extended has space for holding a tennisgear bag. The retriever-docking station is rotatable from a portablecarrying position to a standby position for use during play. Theassembly can also include a lower, outer housing/bumper for absorbingthe impact of a player running into it.

The ball retrieval system optionally includes a ball tracker/locator fordetermining the presence and location of a tennis ball on the court.Video sensors, e.g. one or more video cameras, provide a video datastream output to a processor that calculates the ball location to wherethe retriever is directed. The retriever can then be directed either tocollect another ball or to travel to the ball release location. The balltracker/locator may be positioned on or apart from the retriever, e.g.on a net post.

The invention in ground releases utilizes the forward motion of theretriever to release the ball toward a target. The target can be thecenter service line, or toward a player signaling via the player'sremote control or similar device. The invention's ground releaseembodiments provide a fast and energy-efficient ball retrievalmethodology.

The ball tracker/locator embodiment of the invention tracks and locatesloose balls, and preferably includes a visual or audible indicator tosignal when a ball is located for retrieval (e.g. a ready lightindicator positioned on a net post). The system can indicate any tennisball within its scanned retrieval areas, including balls rolling ontothe court from nearby courts, promoting player safety. Theoperator/player's remote control signals the system to commence aretrieval, manually override a retrieval, or direct the retriever toareas outside the programmed scanned areas.

The retrieval system can include an app on a smart phone or equivalentdevice to establish/initialize the standby position for the retriever,e.g. by the GPS functionality, preferably aside a net post. Theretriever preferably includes an orientation sensor that coordinateswith a programmed-in court grid for establishing retrieval paths andreturns. The docking station may serve as a charging station for theretriever, for example by employing a coil induction-type charger, toextend the retriever useful operating time.

The invention provides an entirely new approach in automating andexpediting play on tennis as well as other courts. Balls are collectedand returned fast and efficiently utilizing the retriever's motion forground releases in the programmed direction, e.g. toward the centerbaseline where a tennis player can easily collect the ball on the racketface. The invention's ground release embodiments are more energyefficient than alternative options, e.g. ball-lifting collectiondevices. Many players favor a baseline singles strategy, thereforereturning retrieved balls there greatly facilitates play. Older playerscan opt to play singles with the assistance of the invention rather thanhaving to resort to the doubles game. Overall, the energy expended inautomatically retrieving a low-mass tennis ball is multiple orders ofmagnitude less than that of a player retrieving the ball. The inventionalso promotes maximum play-time (increasing the play to non-play ratio)to provide better value for the court rental cost, and to reduce matchtime. The latter consideration is particularly significant fortournament and other match play not utilizing ball-persons, for examplein college or other scholastic matches. These matches sometimes employ achair umpire, who can assist in expediting play by controllingretrievals as part of his or her tasks. The invention can decrease thenumber of, or even eliminate, ball-persons, e.g. by obviating thenecessity for stationing one or more ball-persons at the net posts.Another benefit is ball-person safety since ball-persons near the netare occasionally injured by ball strikes.

Yet another benefit is the minimization of human contact with andhandling of the ball, which given the present SARS-Covi-2 (“Covid-19”)pandemic and the transmissibility of viruses via contact with surfaces,presents player safety concerns. The Pro version especially lends itselfto decreasing handling and cross-contamination between matchparticipants and ball-persons conducting ball retrievals at protournaments.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic view showing a standard tennis court layout withdimensions.

FIG. 2 is a perspective view of a ball retriever system positioned on atennis court according to the invention;

FIG. 3 is a side elevation view of a ball retriever according to theinvention;

FIG. 4 is a top elevation view of a ball retriever according to theinvention;

FIG. 5 is a side elevation view of a ball collection blade according tothe invention;

FIG. 6 is a side elevation view of a blade according to the invention;

FIG. 7 is a top perspective view of a docking station according to theinvention;

FIG. 8 is a side elevation view of a retriever with docking station on acart according to the invention;

FIG. 9 is a side elevation view of a blade according to the invention;

FIG. 10 is a top elevation view of a blade according to the invention;

FIG. 11 is a front elevation view of a blade flare end according to theinvention;

FIG. 12 is a top elevation view of a ball ejector-ball feeder/ball rampaccording to the invention;

FIG. 13 is a top elevation view of a retriever according to theinvention;

FIG. 14 is a top elevation view of a retriever according to theinvention;

FIG. 15 is a top elevation view of a retriever according to theinvention;

FIG. 16 is an enlarged view showing expanded details of an interiorsection of FIG. 15;

FIG. 17 is a logic flow diagram according to the invention;

FIG. 18 is a top elevation view of a retriever with details shown inphantom according to the invention; and

FIG. 19 is a top elevation view of a retriever with details shown inphantom according to the invention.

DETAILED DESCRIPTION OF THE INVENTION

DEFINITION(S): “Blade” as used herein also includes structuresresembling those described—e.g. opposing spaced-apart arms and thelike—exhibiting the same functionality and capability of capturing andcontaining a tennis ball in a collection space therebetween.“Processor-controller” includes both (i) an integrated processor andcontroller, as well as a processor component having outputs connected toa controller component's inputs. “Detent” as used herein includes bothi) “active” detent mechanisms that operate by the application of a forcethereto, moving at least one component in order to retain and thenrelease at least one tennis ball in/from a collection space, and ii)“passive” detent structural components that operationally are fixedstructures accomplishing the same functionality. “Remote” or “Remotecontrol” as used herein includes hand-held-type remote control devices,devices such as smartphones with dedicated apps or the like programmedwith the functionalities described herein, and voice command devices,e.g. positioned on the retriever or docking station. “Operator” caninclude players and non-players, e.g. a chair umpire or a ball-person.The phrase “determining the location [of retriever]” includes but is notlimited to establishing the retriever's location relative to a localcourt grid; accordingly, it also includes the use of other systems suchas GPS-based. “Ball” primarily includes a tennis ball but may alsoinclude other [similarly sized] balls used in other racket sports, e.g.(i) a pickleball (≈2.78-2.97″ diameter, weight ≈0.78-0.935 oz) used inthe game of pickleball and played on a court as described above; (ii) aplatform tennis ball, typically either a Vittert-made lo-bounce (V-30)or hi-bounce ball, the latter preferred for certain matches or in thecold; and (iii) a paddle tennis ball which has a relatively lowerbounce/pressure (the paddle ball has an internal pressure about halfthat of a tennis ball). A “fiducial marker” or “fiducial” is an objectplaced in the field of view of an imaging system which appears in theimage produced, for use as a point of reference or a measure. It may beeither something placed into or on the imaging subject, or a mark or setof marks in the reticle of an optical instrument. “Fiducial” accordinglyincludes a ball subject to identification/recognition/print by the robotcamera/image processor during ball retrieval(s) as well as otherinstances noted hereinbelow.

Referring now to FIGS. 2-4, a ball retrieval system 10 includes aretriever 12 having a chassis 14 and mounted thereon at least one drivemotor 16 and a set of wheels 18, wherein at least one of the wheels 18(and preferably two via a connecting axle, e.g. between the rear wheelset) is coupled to the drive motor 16 (the invention also includes eachwheel 18 being associated with and coupled to its own drive motor 16 toprovide all-wheel drive). Retriever 12 further includes a housing 15 onchassis 14, and mounted thereon a means 17 for determining theorientation of retriever 12 (e.g. a 3-axis MEMS magnetometer), a means19 for determining retriever 12 location (e.g. an odometer—each wheelcan include a wheel encoder with an output to a processor-controller 21input, and/or GPS as discussed further below), a means 20 for steeringthe retriever 12 (optionally mounted onto chassis 14 depending ondesign, which are discussed extensively below), a processor 21 (e.g. thePololu A-Star 328PB Micro programmable module with ATmega328PB AVRmicrocontroller) for receiving and processing input signals and withoutputs for controlling retriever operation including steering and driveoutput signals, and an on-board power supply 22 for providing power tothe processor, drive, steering, and other on-board components. It isnoted that although processor-controller 21 is shown as being onretriever 12 with the available on-board localconnections/inputs/outputs, it may optionally be located off-board withthe input-output signals communicated via a wireless communicationsprotocol; e.g. processor-controller 21 might be incorporated into amultifunction device such as a remote control (discussed below) or inanother system processor (also discussed below).

Retriever 12 includes a ball-collector 24 comprising a pair of opposingblades 26 extending from a leading wall 28 of the housing 15 (or chassis14), and which can be integral with the housing 15, e.g. a one-piecepolymer injection mold part; LEDs 29 may also be positioned thereon andare further discussed below. Blades 26 together define a ball collectionspace 30 therebetween for receiving-collecting therein one or moretennis balls, and preferably one or more ball proximity sensors 32positioned on at least one sidewall 31 of an opposing blade 26 adjacentto space 30 (or above space 30 as is further described below) forsensing the presence therein and subsequent release therefrom of theretrieved ball(s). Accordingly, multiple such sensors 32 may be spacedapart laterally along wall 31 as shown for sensing and indicatingmultiple balls collected and released. Sensor(s) 32 may comprise a basicphotoresistor light sensor, an infrared (IR) transceiver, or other basicproximity sensors well known in the art. In a basic version sans bladesensor(s), manual control/direction directs retriever 12 after ballcollection for the subsequent ball release. Blades 26 can be sized andconfigured depending on which of the embodiments herein is utilized, aswell as for the number of balls to be handled at one time, as is furtherdescribed below.

The dimensions of retriever 12's ball collector 24 are based on theofficial tennis ball dimensions established by the International TennisFederation: diameter 6.54-6.86 cm (2.57-2.70 inches), and a mass of56.0-59.4 g (1.98-2.10 ounces). Accordingly, ball collection space 30preferably has a width of at least 2.70 inches extending alongcollection space 30. It is also noted that these dimensions can alsoapply to a pickleball or the like given its similar diameter, which ofcourse can be adjusted (to, say, ≈2.8 up to 3.0 inches given theregulation pickleball range) as necessary. To minimize possible snaggingon the net during retrievals (1¾″ square mesh), blades 26 can be sizedand shaped as discussed below to minimize snagging of in favor a slidingcontact instead.

In one embodiment decreasing the amount of manual control required ininitially directing retriever 12 toward a ball for retrieval, system 10includes a ball tracker/locater 34 comprising one or more balllocation/tracking video sensor(s) 36 and a processor 38 programmed withinstructions for processing a digital video stream from sensor(s) 36 anddetermining a ball location relative to a standard court grid layout.Power supply 39 provides power to system 34 and processor 38. The balltracker/locator 34 can be positioned at, adjacent to, or on a net post40, or on the retriever 12 itself, and is capable of scanning andtracking at least the programmed retrieve areas on each court side.Although processor 38 may also be placed on-board retriever 12, and thepower supplies for each potentially consolidated into one, separatingthese components as shown renders retriever 12 more efficient for itsmain function of retrieving by minimizing its size and weight. Thisextends its range and operating time and provides design flexibility forsystem 34 and its video-processing capabilities. In the embodimentswhere ball tracker/locater system 34 is positioned off-board retriever12, ball tracker/locator system 34 communicates ballposition/directional commands therebetween; accordingly, retriever 12and ball tracker/locator system 34 each preferably includes atransceiver (“T/R”) 43 and 44, respectively, for transmitting andreceiving retrieval commands and other system communications, e.g. atinitiation or completion of a ball retrieve step to input to processor38. In basic embodiments, T/R 43 may instead include just the receiverfor receiving directional and other control commands transmitted byremote 60, while remote 60 likewise may include just the transmitter.Sensor(s) 36 may comprise one or more digital video cameras positionedatop net-post 40 with a field of view covering programmed retrievalareas 46 (dotted lines) on both sides of the net; of course, areas 46are expandable per user preference up to the sensor(s) 36 capabilities.Bullet or fish-eye cameras can expand these, but may increase theprocessing overhead, for discriminating from other objects and noise inball location determinations. An exemplary hardened mountingconfiguration for sensor(s) 36 includes a protective cage 37 to protectsensor(s) 36 from ball or other impacts. The two-camera 36 net postsetup with cage 37 shown in FIG. 2 provides good stereo vision balllocation capabilities and accuracy.

Since the tennis court layout has mirror-image symmetry, processor 38can employ the net as the primary boundary with respect to each balllocation calculation in areas 46. Preferred automatic or manualretriever commands based on area of retriever coverage are optionallyset by a player, e.g. based on individual preferences and matchconditions, via a user interface such as a smartphone app. Accordingly,a player may opt for and set a customized collection range during play;e.g., a player may require additional assistance from retriever 12 as amatch proceeds and the player tires.

As discussed further below, blades 26 are optimally configured to engagein retrievals at or near the net with minimal snagging. Processor 38'sdirection of travel computations/commands preferably direct retriever 12on a retrieval path designed to avoid the possibility of fouling orsnagging on the net or other obstructions. This is accomplished viacomputing a best angle of attack, i.e. the direction retriever 12 takestoward a ball, and processor 38 can include an abort or other routinesuch as alternate path computation upon an on-board proximity sensor 42(in conjunction with sensors 36) sensing an obstruction between it and aball. It should also be noted that if system 34 is positioned on-boardas discussed above, it may alone suffice in lieu of sensor 42 for allball tracking and obstruction sensing purposes, while a more basicversion may dispense with sensor 42 (as well as system 34) altogether infavor of directing retriever 12 to a ball using just the remote. A goodselection for proximity sensor 42 is an ultrasonic-type sensor thatgenerates high frequency sound waves and works on a received echoprinciple to detect an object “interruption”, here the ball. Since itdoubles as a range sensor, processor 38 and/or processor-controller 21can compare its range input with the original ball location calculation(or recalculation as discussed) to differentiate between a ball nearanother obstacle (e.g. the net or net post).

System 10 preferably includes at least one indicator 48, such as alight, for signaling the players that the system has located a ball forcollection, as well as to indicate the location of multiple balls forcollection. Loose balls ready for retrieval include balls rolling ontothe court from players on other courts. Indicator 48 may comprise fiberoptic light indicators or bulbs (incandescent, fluorescent, or halogen)should power, size and space design parameters permit for a givenapplication. Peripherally positioned LEDs (not illustrated) can indicateother system 10 functions as desired, such as multiple balls located andsystem statuses. A player upon observing a “Ready” indication initiatesa ball retrieve operation via a player remote 60 and its transmitter 62(further discussed below), signaling ball tracker/locator system 34'sT/R 44 to initiate the retrieve with processor commands directingretriever 12 toward the ball. Of course, ball tracker/locator system 34may identify/locate more than one ball, and occasionally on both sidesof the net. In the latter situation, system 10 preferably includes adirectional-finding capability built in to the T/R protocols andhardware for ball tracker/locator system 34 to determine which side ofthe net to respond to given that decision tree preference, alsodepending on the actual ball location(s) determined and saved byprocessor 38 for a retrieval directional command.

The ball tracking/locating algorithm can further delineate whether aball is likely to encroach into areas that pose a distraction and/or asafety hazard to a player and then trigger an alert by activatingindicator 48 or to sound an alarm like a player remote vibrate modealert. The invention preferably includes a mapping algorithm that can beset to a player's preference to include a desired retrieval area as wellas to exclude court portions the player opts to self-retrieve, althoughthese may include a player-override should a player opt out or opt in,respectively, to the auto-retrieval. E.g., a player's remote can beprogrammed/configured to permit the player to signal and override theretriever's current or programmed step. The retriever may furtherinclude means visible and/or audible to the player to indicate its readystatus. The functions can be coordinated via programming, e.g. theplayer remote programmed upon a short button press to both transmit a“retrieve” order and activate a system function that visually or audiblyindicates to the player the number of balls identified for collection.The programming can also include a player override to skip one or moreof the signaled ball collections, e.g. by employing another playeraction such as a long remote button press to override a retrieve actionand command the retriever to return to standby. A preferable multi-ballretrieve sequence programs the system to retrieve all identified ballson a side of the court in increments, by collecting/returning theball(s) closest to the retriever base station, with preference given tothose closest to the net, and after retrieving and returning up to itsball-containing capacity, e.g. 2 balls, to then continue to completeretrieval of the rest of the identified balls on that court side untilcompletion or receipt of an override-return-to-base command.

System 10 may include a heuristic-type of learning algorithm thatutilizes the information from each player-signaled retrieval to identifyprobable player-preferred retrieval situations, and provide the playersthe option to switch system 10 to an automatic mode not requiring aplayer retrieval signal (but optionally subject to override as discussedherein) whereby system 10 engages in retrievals automatically. In theautomatic mode, system 10 can utilize either indicator 48 or an audiblealert to indicate to the players that retriever 12 is commencing orabout to commence a retrieval, with perhaps a design time delay to allowplayer override via a remote.

Retriever 12 upon collecting the ball is then directed to the calculatedrelease point by processor 38 (although this can alternatively beconducted by processor-controller 21 or by the direct signaling linkupbetween player remote 60 and retriever 12). Except for those versionscomprising an ejector or the like, the ball release is accomplishedsolely by the motion of retriever 12 toward the target (center servicemark, player, etc.) and the matching forward velocity of the ball.Retriever 12's acceleration or inertia (along with the detent whenprovided or actuated) maintains the ball in collection space 30 untilretriever 12's programmed deceleration releases (or ejects as discussedbelow) the ball at the release point from collection space 30, rollingthe ball to the target to complete the ball return.

Upon release/ejection of a ball or balls from space 30 in the programmed(or user-override) direction, processor 38 (or processor-controller 21in other configurations) is programmed to direct retriever 12 to returnto the standby off-court position, preferably adjacent to one or theother net posts, to await the next retrieval command. Processor 38preferably includes programming instructions that upon executioncycle/turn off indicator 48 once a ball is captured as sensed bysensor(s) 32.

Ball location/tracking video sensor(s) 36 and on-board proximity sensor42 preferably cooperate during a ball retrieval where system 10 isprogrammed with instructions that upon execution “hand off” the balllocation functionality from sensor(s) 36 to sensor 42 upon the latteracquiring a ball within a design or programmed distance, whereuponretriever 12 tracks the ball via sensor 42 up through ball capture asconfirmed by sensor(s) 32. Sensor 42 upon retrieving/collecting thenworks in tandem with sensor(s) 32 where, after inputting a signal tosystem 10 confirming a ball collection, system 10 then transitionssensor 42 into an obstacle detector function; in other words, dependingon the retriever 12's calculated route status, system 10's logic andprogramming instructions upon execution cause sensor 42 to cycle into anobstruction avoidance/remediation subroutine, or during the finalprogrammed ball release step, optionally engage a ball releasesubroutine for a preemptive ball release prior to the defaultprogrammed/mapped release point upon sensor 42 sensing a player closerin than the default ball destination target such as the center baseline.Upon ball collection, system 10 preferably updates the actual ballcollection location since the ball position may have changed due to windor other factors, or even error in the original path computation that isaddressed upon acquisition by and hand-off to sensor 42. This actualball collection location is then factored into an updated retriever 12path calculation, for purposes of updating the retriever 12direction/release path from that position, as well as updating theretriever 12 return path to its standby position.

Prior to play in the initial setup of system 10, the programmedinstructions can include a system calibration check/initializationwhereby the players can confirm system 10's ball location accuracy, e.g.by positioning the retriever 12 in its standby location to lock it in,and by confirming the accuracy (and adjusting if necessary) of sensor(s)36; one way is by placing a ball in one or more set locations to confirmsystem 10's accuracy vis-à-vis the programmed court grid coordinates.

System 10's ball location and path calculations may yet be limited inprecision by the accuracy and sensitivity of locator 34/sensor(s) 36.Means 19 for determining retriever 12 location registers retriever 12'sdistance of travel and/or location coordinates (e.g. relative to aprogrammed-in court grid) during a retrieval, which information is inputinto real-time path computations to guide retriever 12 along thecomputed path to the computed ball location (except in the more basic,remote-only operated embodiment discussed herein). Upon handoff tosensor 42 and any path course variation from the calculated original,the revised course calculation updates the ball retrieval/retriever 12location, which data is utilized for both the ball return calculationand when necessary the retriever 12 return path to the standby location.In order to eliminate or at least minimize additive path error effects,i.e. over multiple retrievals, the return of retriever 12 to the standbylocation after ball release may consist of an independent actioncontrolled by direct communication with a docking station or anet-post-based R/T and/or system 10 is reinitialized after eachretrieval or path circuit to eliminate path error accrual (all discussedbelow).

Retriever 12 in one embodiment can include a retrieval path routine thatutilizes the court lines (singles, doubles, baselines, service lines) toguide its return-to-standby/docking position (and/or to employ fiducialsas discussed herein). For example, retriever 12 after the ball releasecan move in reverse and upon reaching the doubles line closest to thenet post standby location track along that line back to the net post tofacilitate the accuracy of the return path. An optimized retrieval pathmay be calculated from the singles line detection/arrival and then backto the standby/docking station location (e.g. a curved path). Retriever12 can therefore include a line tracker (e.g. mounted on the aft end)programmed with instructions to process the line tracker data input andupon execution control the steering to direct the retriever along theoptimal calculated path, e.g. toward the doubles line and then to thestandby/docking station location. An exemplary line tracker is theTCRT5000 IR sensor including a TX and RX inside. If a bright colorsurface is detected in front of the sensor, such as a white paper (about1 cm away), most of the IR rays will be reflected; if a dark color likeblack surface is detected, most of the rays will be absorbed, while onlya small amount will be reflected. The RX will output different analogsignals according to the intensity of the reflected IR light.

Programming and computational efficiency can be optimized by utilizingthe retriever orientation determining means 17 (e.g. a magnetometer) inconjunction with both the distance-location computation-determinationand the line tracker; e.g. upon ball release, retriever 12 can realignparallel to (or at a designated angle) to the initial standbyposition-magnetometer axes-vectors and then proceeding (preferably inreverse as discussed) toward the sidelines closest to and extendingapproximately from the standby net post-docking station, and the linetracker sense the line(s) with retriever 12 programmed to turn (@90degrees or the necessary amount) and then utilize the doubles sidelineto return toward standby. A proximity sensor (sensing the distance forthe retriever 12 in approaching to and resuming the desired standbyposition) in conjunction with the other docking means discussedre-establish the desired standby position. Of course, there may beoccurrences of mis-retrievals during the operation of system 10, e.g.due to retriever 12 encountering an obstacle, snagging, failing tocollect a ball in a directed location due to various factors includingsystem malfunctions, or intervening factors such as wind. System 10therefore includes means for correcting a mis-retrieval, e.g.programming instructions that upon execution stop retriever 12 andredirect retriever 12 on a course to alleviate the issue, e.g. to thenreverse (or attempt to reverse) direction when there's a failed handoffto sensor(s) 32 or some other issue, after which a further instructionand command retriever command may be to resume the current retrieval butwithin a set time interval, otherwise, stopping to await further commande.g. a player-initiated override, or returning to the standby location(e.g. outside a net post). One such routine applicable to a retrievalalongside the net would instruct retriever 12 to stop, back up aspecified distance toward or along the original line of approach, andthen correct the programmed path a specified degree or vector to attemptto clear the snag. The programming instructions can include an automaticabort routine that may be subject to player manual override (such as a“kill” button, an input sequence mated to the main player remote controlswitch/button, or voice command (e.g. “Stop”)). The “retrieve fail”override preferably alerts a player visually, audibly, tactile (e.g. ahaptic vibration mode or the like incorporated in the player remote), orvia a player-selected preference. Override protocols may further includea “release fail” protocol upon sensor(s) 32 indicating that a ball orballs appear to be stuck in and fail to timely release from space 30.

Retriever 12 includes detents 50 (e.g. on walls 31) for retaining andreleasing a ball to ensure a ball is not lost from space 30 prior to itsprogrammed release, e.g. when retriever 12 undergoes changes in itsforward motion velocity component during turns or deceleration. Detents50 can be passive structures, active components, or a combination ofboth, and dampen (passive) or restrict (active) the ball's forwardmotion component in order to retain the ball in space 30 until theretriever 12 forward motion component again equals or exceeds that ofthe ball. Exemplary passive detent 50 is illustrated in FIG. 5 (andFIGS. 9-10, described further below). Blade 26 (the opposing blade 26being its mirror-image structurally, although there may be minorvariations in some embodiments, e.g. slots for sensors, fasteners, orthe like) includes a concave (with respect to space 30) area 51 onsidewall 31 for urging a ball into (or back into) space 30 duringcollections, retriever turns and the like to retain the ball up to theprogrammed (or player/operator-initiated) release. Passive detent 50 mayfurther include an upper lip 56 and/or a stop 58 (e.g. a ridge orbumper) to help retain and urge the ball back into collection space 30but also configured to not interfere with the ball release. For example,proper clearance must be provided between opposing stops 58—theirplacement height vis-a-vis the ball circumference can thus bedeterminative. Stops 58 may be contoured (e.g. concave toward the aftend) to deflect the ball back into space 30.

System 10 (or retriever 12's processor-controller 21 in the on-boardembodiments) with or without a passive detent 50 configuration may beduly programmed to hold or attain a desired retriever 12 forwardvelocity component during retrieval for best results, e.g. viaprogramming instructions that upon execution coordinate the inputs fromsensor 42 and sensor(s) 32 to identify a ball exiting space 30prematurely and respond by adjusting retriever 12's velocity anddirection (and also utilizing sensor 42 if present to track the ball forre-retrieval).

FIG. 6 illustrates active detents 50, consisting of a retractable detentpositioned on at least one sidewall 31 of each blade 26. Exemplaryactive detents 50 include solenoid-actuated pistons or buttons or ahinged blade 202 (e.g. a flipper) on (one or both of) a leading end 204of blade(s) 26 responsive to ball retention/release system commands andthe like. Unlike the passive detent discussed above, the active detentsare sized to interfere with a ball release, i.e. these extend into andretract out of the ball release path. Although this adds to the designcomplexity of retriever 12, it simplifies programming by eliminating theproblem of retaining the ball in space 30 otherwise addressed by systemprogramming and input based on the sensors as described above. It alsosimplifies system programming by providing more flexibility inprogramming retriever 12's locomotion since it provides more latitude inretriever path computations and decreases path re-computations.

The retriever may further include ejector 52 mounted axially (e.g.toward the aft end of space 30 and along the retriever body centerline(“C/L”) as illustrated in FIG. 4) for ejecting a ball or balls atrelease. Exemplary ejector 52s include (i) the solenoidal type, with thepiston striker mounted orthogonally to impart a vertical motion to theball, and (ii) a flipper, e.g. a lever arm-spring-operated device, toname but a few, triggered by a release command from processor 38 toeject a ball in the desired or program-computed direction at either asuitable, a designated, or a processor-computed velocity. Unlike atennis ball, a pickleball, platform tennis ball, and paddleball and thelike are not amenable to being readily scooped (or bounced) up from theplaying surface given their reduced-bounce propensity and the shorterpaddle versus racket. Paddle tennis is often played in the cold, andwith a ball that even using a high-bounce version (e.g. the V-30)requires a respectable warm-up to bounce sufficiently.

Retriever 12 with an ejector 52 is therefore particularly advantageousfor paddle-type court games, especially given the effort for the playersto constantly bend over to pick up balls. Ejector 52 in these non-tennissports presents less risk of harm to a player if struck upon ejectiongiven the types of balls; for example, a pickleball weighs under 1 oz(≈0.9 oz or 25 g) and a tennis ball more than double that (≈2 oz or 58g). In the (ii) flipper version, the lever arm assembly is apivotally-mounted “flipper”-style mechanism as shown, whereby withretriever 12 preferably positioned aft end toward the target where withthe flipper mounted as shown, ejector 52 ejects the ball in the targetdirection. Paddle and platform tennis balls can also be safely ejectedin this manner. Retriever 12 may include a user setting to select either(i) eject-only or (ii) both ground and eject release. Of course, it maybe desirable for safety reasons to configure on-board indicator 65(described below) to signal a player when (ii) is opted toeject-release, visibly or audibly.

Absent ejector 52, retriever 12's velocity at the time of ball releasealone dictates the ball release exit velocity; in this design, processor38 can include programming instructions that upon execution computes apreferred ball release exit velocity based on the ball release location,preferably subject to user adjustment. For example, a ball locatedcloser to the baseline or return target may dictate a slower releasespeed and a closer release point (with respect to the baseline/returntarget) than a ball located more distant. Court surface type may alsodictate these parameters.

Ejector 52 may optionally be capable of imparting a vertical (y-axis)directional component (as well as a horizontal (x-axis) directionalcomponent, although in other embodiments the forward motion of retriever12 may alone suffice for this purpose). E.g. ejector 52 may comprise aspring-tension-biased lever arm (a “ball-flipper”) configured to accepta rolling ball over its leading edge onto a concavity sufficient toreleasably hold the ball, with ejector 52 responsive to a ball releasecommand to eject and propel the ball to a height, e.g. 2-4 ft.,sufficient to enable a player to catch the ball or collect it on theracket face or on the bounce with minimal exertion. This option providesa low-power mechanism as compared to alternative ball capture-releasedevices for achieving the same result, and it may also utilize retriever12's momentum to impart a desired x-axis component and eject the ball inthe programmed direction.

Ball ejectors 52 include as stated previously a solenoid, or “firingram”-style ball launcher such as the electrically powered launcherremotely controlled to actuate linear movement of a striking armejecting the ball as is described in U.S. Pat. No. 6,571,743 issued Jun.3, 2003 (W. Curry); a fly wheel launcher such as that described in U.S.Pat. No. 7,691,012, issued Apr. 6, 2010 and U.S. Pat. No. 8,287,404,Oct. 16, 2012 (Cucjen et al.); and a hammer-style that relies onrotational movement of a striking arm rather than linear motion, asdescribed in U.S. Pat. No. 6,176,230, issued Jan. 23, 2001 (R.Thompson), and in U.S. Pat. No. 9,339,716, issued May 17, 2016 (E. Ward)which also describes automatic actuation triggered by a ball sensor, afeed funnel, an ejection chute, and adjustability capability for bothdistance and ball launch angle; all of which patents are incorporatedherein by reference. Embodiments of the invention incorporating some orall of these functionalities are included in the scope of the inventionas is further discussed herein, although given the specific nature ofthe present invention versus the divergent applications of the citedreferences, it is desirable as discussed herein to employ a design inthe invention that optimizes energy efficiency and minimizes weight.

Processor 38 (e.g. the Pololu A-Star 328PB Micro programmable modulewith ATmega328PB AVR microcontroller) is programmed with instructionsthat upon execution calculate a loose ball location and movement basedon the video stream data relative to the standard court dimensions andthe retriever 12 position and that guide retriever 12 for an optimalintercept, which computations can change depending on changes in ballposition, direction and velocity. The angle of intercept of retriever 12(and thus the retriever 12 computed path) is preferably optimized forsubsequent release in a programmed direction (as well as for additionalball collection in the multi-ball collection scenario). The programmedball return direction is preferably toward the center service mark onthe same court side, that is, the center of the baseline. The returndirection could alternatively be set in the direction of a signalingplayer via a “homing” signal means incorporated into the remote control.Homing signal means may include a directional RF locator, a pair ofreceivers or transceivers located on opposite sides of retriever housing15 that are selectively responsive to a signaling device, or a uniqueremote device identifier with a device tracker to ascertain on whichside of the court the remote is located. In addition, voice commandsdiscussed herein can provide elements of this functionality.

System 10 is preferably configured to optimize ball retrieval efficiencywhile minimizing the processor 38 overhead. Accordingly, in thepreferred embodiment the video-scanned and image-processed area isprimarily directed to the on-court areas 46 affording the optimal timeand effort savings to the players, starting from the net and proceedingback toward the service lines. As discussed, the player remotespreferably include an override function that enables a player to directthe retriever to positions outside the programmed areas 46 to manuallydirect the retriever in those instances. The remote can include sensors,e.g. gyro, accelerometer, magnetometer, etc. that sense the 3-axisorientation of the remote, which can be accessed by system 10 andretriever 12 to interpret the corresponding motion of the remote indirecting the retriever 12 toward an off-court ball as a sort of“tractor beam” and even impart a Wiimote functionality to facilitatedirecting the retriever, further discussed below.

The use of more than one remote in directing a given retrieval canimprove the accuracy of a retrieval path, whereby signals from two ormore retrievers located on different parts of the court indicating thelocation of a ball can triangulate or otherwise improve thedetermination of more precise coordinates to which the retriever isdirected. For example, in a professional tournament, more than oneball-person/ball-kid may each hold a ball retrieval direction-indicatingremote to optimize the retrieval performance of one or more retrievers,including a retriever array as discussed herein.

Upon a player via remote 60 directing retriever 12 on a manual path foran out-of-area 46 retrieval, system 10 in this embodiment hands off tosensor 42 for the ball location input for retrieval and then upon sensor42 sensing and signaling the ball, system 10 assumes control from remote60, calculating a revised retriever 12 return path to the ball returnlocation and directing retriever 12 there.

As discussed above, processor 38 is preferably optimized with respect toits overhead (and consequently its cost and power requirements) todetermine the location of a ball efficiently while minimizingcomputation time. Sensors 32, 36, and 42 are selected for accuracy andsensitivity to coordinate effectively for ball collections and releases.

A tracking/location technique useful in the present invention isdescribed in Wang, preferably with modifications. Wang describes atennis ball tracking system capable of detecting a ball up to 5-6 metersfrom its robotic-based platform (i.e. “on-board” as the term is usedherein), a range that can be expanded utilizing a bettersensor-processor combination. Wang employs a PandaBoard as its mainprocessor, functioning to conduct “compute-intensive tasks and externalcommunications” via a dual-core 1 GHz ARM Cortex-A9 processor (“roughlyequivalent to a netbook in terms of computing power”), and whichincludes an onboard WIFI module, Ethernet, and 2 USB ports. The ArduinoATmega2560-based development board, used for low-level motor and sensorinterfacing, is clocked at 16 MHz with 8 kB SRAM, 4 hardware UARTs, a2-wire interface for I2C, and 16 ADC channels. The board communicateswith the Pandaboard via a hardware serial port on the OMAP processor.Wang states that the hardware serial is important because USB-serialincurs a non-deterministic latency from the USB-serial driver stack in anon-real-time operating system, which may result in random localizationerrors and destabilize the control loop. Wang employs stereo vision,with two PlayStation Eye cameras mounted in a parallel stereoconfiguration and interfaced with the PandaBoard via USB. The camerasprovide a framerate of 100 frames/s at a 320×240 resolution and includean inexpensive and relatively low-latency Linux driver.

The Arduino Mega board is easily reprogrammed via USB and providesflexibility in interfacing different attachments or sensors. One sensorsuite features a quadrature optical encoder rim and a 9 DOF IMU. 1)Wheel Encoder: The optical encoder is used both for velocity control andfor odometry. It uses an Omron EE-SX1031 slot-type dual photointerruptorand an appropriately-sized codewheel to provide a resolution of 204counts/m. 2) IMU: The IMU module used is the commercially availableSparkFun 9 DOF IMU. It provides stable, filtered yaw rate measurementsfor odometry. The IMU is mounted on the centerline of the car foraccurate yaw measurements. It is not drift-free, but it has provenreliable enough for incremental measurements of odometry. The ArduinoMega is programmed with software necessary to interface with the motors,servos and sensors. It runs a message-processing loop to receivecommands (velocity, steering angle, ball pickup state) from thePandaBoard. The Arduino Mega runs a few time-critical tasks. Foremost ofthese, it runs a PID control loop to set the speed of the robot. This isdone on board the low-level microcontroller because the control looprequires precise timing stability. It also sends odometry readings backto the PandaBoard at regular intervals. This ensures that the readingsare spaced evenly so that the time delta between readings is consistent.

The invention in most embodiments avoids certain complexitiesnecessitated by on-board court navigation approaches such as Wang bypositioning video capture sensor(s) 36 off-board, adjacent to the courtand as discussed above preferably on the net post covering thepre-calibrated collection areas and/or those set by the user. Atstartup, system 10 may be initialized (or “zeroed”) by placing a ball ata predetermined location, e.g. at the junction of the center service boxlines (the “T's”) on each side, with a system 10 startup subroutinecarrying out initialization based on relating the video data from eachposition to “lock in” and establish those coordinates the programmed-instandard court grid position and dimensions, optionally with GPScoordinates if system 10 is so equipped. Retriever 12 is concurrentlypositioned outside a net post and its starting (“standby”,waiting-to-retrieve position) may be similarly initialized at systemstartup. As opposed to Wang and similar on-board approaches that directprocessor overhead to determining line placement, e.g. via particlefiltering, for device navigation, these embodiments of the inventionavoid this complexity. Furthermore, the invention by positioning thesensor 36 separately from retriever 12 avoids other complexities ofprior art on-board path motion and control systems, e.g. the falsepositives and errors introduced by camera sensor vibration.

As noted, system 10 when GPS equipped can apply it to bothinitialization protocols at startup to establish/initialize the standbyposition for retriever 12 as well as the court grid relative to its GPScoordinates and orientation since different tennis courts have differentcompass orientations. Retriever 12 can include on-board GPS in its T/Rto facilitate location determinations and path computations. Remotecontrol device 60 may also be GPS-equipped; it may also utilize a smartphone with invention protocols programmed into a dedicated app in IOSand/or Android with a compatible GUI, encompassing the control andoperating functionalities described herein. For example, aninitialization procedure may have a player initialize retriever 12'sstandby location, then take position at the center service-mark line andtag that location to the system court grid map and GPS coordinates, tothus establish the overall GPSed court map dimensions which are alsolinked with the retriever 12 position. If necessary, repeating this stepon the opposite side of the court completes the specific mappinginitialization sequence, although retriever 12's orientation relative toeither post should alone suffice without this additional step. Ofcourse, there are equivalent ways to establish the specific map, this isto describe just one such methodology. Also, the on-board sensors,including GPS and/or orientation means 17 can establish and directprecise positioning of retriever 12. Design choices can be based on thefollowing considerations.

Navigation/Positioning Sensors

Positioning sensors are used to approximate the position of a robot,some for indoor positioning and others for outdoor positioning. The mostused positioning sensor is a GPS (Global Positioning System). OrbitingGPS satellites transmit the signals and the retriever GPSsensor/receiver acquires and processes these signals. The processedinformation can be used to determine the approximate position andvelocity of a robot, but these can function better outdoors and can bepricey. A digital magnetic compass provides directional measurementsusing the earth's magnetic field for direction determinations. Thesesensors are cheap compared to GPS modules, but a compass works bestalong with a GPS module if you require both positional feedback andnavigation. Philips KMZ51 is sensitive enough to detect earth's magneticfield. Localization refers to the task of automatically determining thelocation of a robot in complex environment. Localization is based onexternal elements called landmarks which in the present invention can bepart of the initialization procedure as discussed using ball placementin specified locations.

Sensor(s) 36 may be positioned at either net post 50 given themirror-image symmetry relative to the court grid of either placement.This is especially advantageous since outdoor courts are usuallyoriented N-S so that the sun is not directly in a player's eyes when thesun is low in the sky. It follows that the selection of net postplacement may be according to the sun's position, in that it isgenerally preferred for optimal object tracking/resolution/imageprocessing that the sun is behind the sensor(s) 36.

Many indoor court facilities provide courts compartmentalized from oneanother via heavy vinyl, fabric or composite, curtains behind eachbaseline and side curtains that are often mesh curtains slidablysuspended from overhead cables for ease of opening and closing toaccess/close off the court as needed. Retriever system 10 preferablyincludes a user-selectable setting for selecting a programmed-in indoorcourt layout included in the processor programmed instructions includingthe parameters associated with the indoor layout/dimensions, whileretriever 12 as discussed above preferably includes one or moreproximity sensor 42 (mounted where most effective) capable of sensingside obstructions or clearances to minimize interference, e.g. fromindoor court side curtains, by signaling processor-controller 21 todirect or redirect retriever 12 as needed. A dedicated smart phoneapp/GUI that includes assigning a private comm protocol to just thoseplayers/that court minimizes interference between courts and the use ofmultiple retrievers in the vicinity by different courts and players.

Processor 38 (that is used in most embodiments) is programmed to processthe video data and determine a probable ball location. In oneembodiment, the invention incorporates the approach described in Wangfor locating stationary balls on the surface of a tennis court. To cutout impossible regions where stationary balls could be found and reducethe search domain, only the region below a finite horizon extending overthe court length is considered. Although the described approachaddresses at most one ball being present in the frame, given theemployment of an on-board sensor described further below, the inventioncan collect more than one ball captured in a frame. Textures and noiseare smoothed by running mean shift like clustering over space, e.g. asdescribed in D. Comaniciu and P. Meer, “Mean shift analysis andapplications,” in Computer Vision, 1999. The Proceedings of the SeventhIEEE International Conference on, vol. 2, 1999, pp. 1197-1203 vol. 2.The resulting posterized image has larger areas with nearly constantcolors. Connected components or blobs, which are contiguous regions ofnearly constant color are generated from the posterized image using aflood fill algorithm. Connected components which have pixel area muchgreater than what is possible for an image of a taken ball from thecamera's height are discarded.

Contours bounding the connected components are found, e.g. as describedin S. Suzuki et al., “Topological Structural Analysis of DigitizedBinary Images by Border Following,” Computer Vision, Graphics, and ImageProcessing, vol. 30, no. 1, pp. 32-46, 1985,http://www.sciencedirect.com/science/article/pii/0734189X85900167.Contours help in obtaining useful geometric properties like area, shapeand position of the blobs. Contours are filtered on size and shape. Thisfiltering based shape is done by first fitting ellipses to the blobsusing a least squares technique, e.g. as described in M. Pilu, A.Fitzgibbon, and R. Fisher, “Ellipse-specific direct least-squarefitting,” in Image Processing, 1996. Proceedings., InternationalConference on, vol. 3, September 1996, pp. 599-602 vol. 3., and thenevaluating the following two measures,

ρ:=M/m,

where M=length of major ellipse axis, m=length of minor ellipse axis,and

Δ:=1−Area(Blob)/Area(Ellipse),

and where for a circular blob, ρ→1⁺ and Δ→0⁺. The remaining contours'pixels are then converted to HSV color space for color-based filteringaided by the tennis ball's distinct optical yellow color. In the rarecase that multiple blobs remain, the one with the largest area isassumed to represent the ball. Successive averaging and filtering leadsto a progressively diminishing set of ball candidates which aids inreducing the computation overhead, a significant consideration in thesystem 10 design as noted and can detect balls at least up to 5-6 metersdistance from sensor(s) 36. Wang states its dual-core 1 GHz ARMCortex-A9 processor as roughly equivalent to a netbook in terms ofcomputing power, whereas current equivalent processors are more compact,less expensive, and more powerful; furthermore, Wang positions theprocessors on-board, whereas in the preferred embodiment of theinvention it is position Single object tracking, in which a target isoften initialized manually in the first frame and then is tracked andlocated automatically in the subsequent frames, is a hot topic incomputer vision. The traditional tracking-by-detection framework, whichoften formulates tracking as a binary classification problem, has beenwidely applied and achieved great success in single object tracking.However, there are some potential issues in this formulation. Forinstance, the boundary between the positive and negative trainingsamples is fuzzy, and the objectives of tracking and classification areinconsistent.

It should also be understood that there are several alternative computervision-based tracking approaches also within the scope of the presentinvention. For example, single object tracking, in which a target isoften initialized manually in the first frame and then is tracked andlocated automatically in the subsequent frames, simplifies system 10 inthe sense that one ball at a time can be tracked and its positiondetermined without having to track another ball either simultaneously orwhile the tracking focuses on a first ball being tracked. This isbecause the tennis match play is briefly suspended in-betweenshots/serves. Each ball location is stored in processor 38 memory(and/or processor-controller 21 memory in some embodiments discussedherein) and accessed for retrieval, either one-at-a-time or in amultiple ball batch-style retrieval cycle/retriever 12 operation. Ofcourse, other factors including environmental, e.g. wind moving the ballin the outdoor setting, may intervene; system 10 can therefore includean optional anemometer 53, preferably integrated with the ball locator34 and processor 38 modules and comprising a MEMs-type device, e.g. theModern Device model MD0550 wind sensor Rev. C, to sample real-time windvelocity and direction to input into the retriever 12 directionalcommand structure a correction factor for directional calculations andcommands for retrievals.

Locator system 34-processor 38 employ ball tracking technology as noted.Traditional single-object tracking-by-detection framework, which oftenformulates tracking as a binary classification problem, has been widelyapplied and achieved great success in single object tracking. Someissues with this exist. For instance, the boundary between positive andnegative training samples may be fuzzy, also, the objectives of trackingand classification may be inconsistent in some applications. There are,however, approaches that address these aspects.

In one approach, tracking is addressed as a fuzzy classification problemand measures the importance of training samples by assigning differentmemberships to and provides stricter spatial constraints. It alsoapplies a fuzzy least squares support vector machine (FLS-SVM) logic toimplement a concrete tracker. It analyzes the primal form, dual form,and kernel form of FLS-SVM and derives the corresponding closed-formsolutions for efficient realizations. Then, a least squares regressionmodel is built to control the update adaptively, retaining therobustness of the appearance model. The reduction in the requisiteprocessing overhead extends system 10's operating time and retriever12's range between recharges.

MathWorks's Computer Vision System Toolbox™ also provides video trackingalgorithms, such as continuously adaptive mean shift (“CAMShift”) andKaneda-Lucas-Tomasi (KLT). These algorithms are useful for tracking asingle object or as building blocks in a more complex tracking system.The toolbox also provides a framework for multiple object tracking thatincludes Kalman filtering and the Hungarian algorithm for assigningobject detections to tracks. For example, pixels detected using“vision.ForegroundDetector” separate moving objects such as a ball fromthe background. The background subtraction, however, may only locatepart of a ball because of the low contrast between the ball and thecourt. Also, two other issues include the region's center is usuallydifferent from the ball's center, that is, there is an error in themeasurement of the ball's location; and, the location of the ball is notavailable when it is occluded by another object, e.g. a portion of thenet. These issues are addressed using the Kalman filter, to eliminatenoise present in the tracking system and applying its use predict methodby itself to estimate the ball's location when it is occluded by anotherobject. When the ball is detected, the Kalman filter first predicts itsstate at the current video frame, and then uses the newly detectedobject location to correct its state. This produces a filtered location.When the ball is missing, the Kalman filter solely relies on itsprevious state to predict the ball's current location. The Kalman filteris configured with a set of suitable configuration parameters, e.g.using the “trackSingleObject” function and other options in the“configureKalmanFilter” function which returns a Kalman filter objectbased on provide five input arguments. The MotionModel settingcorresponds to the physical characteristics of the object's motion andis set to either a constant velocity or a constant acceleration(/deceleration) model. The invention preferably sets to the latter,given that the ball is subject to constant deceleration due to rollingresistance/friction (unless as noted some environmental factor ispresent such as wind pushing the ball when outdoors). Of course,typically objects do not move with either constant acceleration orconstant velocity. Kalman filtering therefore includes a MotionNoisesubroutine to allow specifying the amount of deviation from theprogrammed motion; increasing the motion noise parameter commands theKalman filter to continue processing additional incoming video objectlocation and velocity tracking data to refine the ball locationdetermination. This subroutine can further be paired with data fromanemometer 53 to refine a ball location determination, or absent that,adjusted by a GUI or other such system 10 user setting option.

The Kalman filter advantageously also includes a routine/protocol toaddress multiple object tracking. Applying “vision.KalmanFilter” objecttogether with “assignDetectionsToTracks” function addresses assigningdetections to tracks, determining whether a detection corresponds to anew object, and in the case of an occluded single object can separateobjects in close mutual proximity.

System 10 therefore can start processing the video data relevant to agiven ball retrieval commencing with resolving a first target objectframe, typically when a ball has slowed to a resolvable object speed.This fits perfectly within the parameters of the tennis game, since aplayer would not instantly recognize a desired pause-for-retrievalsituation either until assessing the ball's direction and speed, andthen estimating or “guesstimating” its probable path/destination. Bythen, system 10 will have detected and “locked in” the ball, along withits speed and its path, with processor 38 calculating the optimalretrieval path for intercept and return (and subject to correction and afinal location determination as discussed). To simplify the programming,processor 38 can be programmed to commence with a zero-velocity statefor a first video frame detection at the net, with sensor(s) 36positioned to scan the programmed areas 46. A ball hitting the net andits motion stopped thereby will thus enter the field of view with itsvelocity initially zeroed, simplifying the video processing required.

In another approach, sensor(s) 36 can comprise a pan-tilt-zoom (PTZ)camera using a conventional, off-the-shelf webcam for object tracking,but as this necessitates a motor-assembly, it does involve additionalcomplexity/cost and increases the power overhead. In an exemplaryapplication, it may utilize a conventional Fast Compressive Tracking(FCT) algorithm with capabilities including programmed learning, fastcomputation and robust performance, and can further include a programinterfaced with embedded system through serial RS232 to command thecamera motor(s) for centering the image in the current frame. Theprogramming instructions can then interface with the directional commandto retriever 12 to direct it to a ball location with improved precision.

The invention in some embodiments includes the utilization of RobotOperating System (“ROS”)/Ubuntu programming protocols that utilizecamera imaging using open-source programs and tool suites, which includefiducials or fiducial markers (defined above). A fiducial marker orfiducial is an object placed in the field of view of an imaging systemwhich appears in the generated image for use as a point of reference ora measure. It may be either something placed into or on the imagingsubject, or a mark or set of marks in the reticle of the opticalinstrument. “Fiducial” may be viewed here in a broad sense, so as toinclude identification/recognition/print by the robot camera/imageprocessor of not just a ball when preparing for/commencing ballretrieval(s), but also other fiducials, including a hand, racket/racketface, and player gesture that in combination with programmedinstructions upon execution direct a ball release. Similarly, fiducialscan include a marker or device attachable to a player or player'sequipment for selectively triggering robot responses (optionallyinterfaced and GUI-programmed for flexibility of use). TheUbuntu-fiducial protocols can be programmed to interface with the voiceand command protocols discussed herein as desired.

Other fiducial applications may include the use of multiple cameras onretriever 12 or elsewhere for providing spatial differentiation withfiducials located at select spatially-differing positions (e.g. relativeto the vertical ‘y’-axis), eliminating the need for one or more servomotors in a TPZ (tilt-pan-zoom) using just a single camera coveringmultiple fiducials. This setup could decrease servo-motor noise, andwear and tear, with system complexity essentially unchanged albeit usingother, substitute components. For example, capturing, processing, andprinting an image of a loose ball fiducial on the court surface occurson a lower imaged x-z plane than a higher fiducial—e.g. the player's armor racket. Accordingly, to reduce/eliminate pan-tilt/zoom servo motoroperation noise in the retriever, positioning and utilizing 2 or morecameras on the retriever, net post, or another desired location orcombination thereof, and focused on spatially-differently-positionedfiducials, is within the scope of the invention.

As discussed above, retriever 12's ball proximity sensor 42 (included onsome embodiments of system 10) complements ball tracker/locator 34 whenthe latter is positioned off-board and for ball collection outside itsrange (or in the on-board basic configuration as the sole ball locator).As discussed above, system 10 includes a player remote control device 60with a transmitter (or transceiver) 62 for transmitting balltracker/locator 34 system override/retrieve commands. Remote control 60may be configured like a Wiimote to direct the retriever 12 in thedesired direction of travel toward an off-court or other ball simply bymoving and rotating the remote, as further discussed below. In themulti-sensor directional handoff system 10 configuration discussedabove, upon arriving in proximity to the ball and within range of sensor42 directional control of retriever 12 is handed off to process theinput from sensor 42 for ball retrieval and ball return it to thesignaling player, then the return to standby location. Since it isanticipated that there can be retrievals when retriever 12 encounterssome kind of off-court obstruction as discussed above, sensor 42 iseither capable of differentiating between a ball and an obstruction oralternatively as discussed above system 10 programmed to abort aretrieval operation in progress, e.g. by activating its “retrieve fail”protocol.

The Wii Remote-style configuration and control can be implemented usingtechniques such as Richard Ibbotson's Wiimote Bluetooth code (Appendix‘A’, infra). For example, it has been modified to control roboticdevices and an RC car. The Wiimote senses acceleration along three axesusing an ADXL330 accelerometer and utilizes a 16 KiB EEPROM chip fromwhich a section of 6 kilobytes can be freely read and written by thehost. The Wiimote using two AA batteries can operate for up to 60 hourswhen using just its accelerometer functionality. Other devices such asthe Yuneec Wizard Wand, a one-handed positional drone controller,provide similar functionality. Changing the angle of orientation of theremote control controls the devices' locomotion and/or 3-D movements viaWIFI.

Operationally, such remote devices enable the translation of the devicemovement into commands to a user interface, in system 10 via WIFI suchas Bluetooth communication protocol. The device's sensors' x-axisattitude (roll), y-axis elevation (pitch) and/or z-axis heading (yaw)motion of the device are translated into directional steering commandsof retriever 12. In addition, some exemplary embodiments of the presentinvention can also measure linear movement of the device along the x, y,and z axes to generate user interface commands.

Multiple combinations of sensors can be used to achieve the desiredcontrol and functionality. For example, two rotational sensors and oneaccelerometer 506 can be employed. The rotational sensors can, forexample, be implemented using ADXRS150 or ADXRS401 sensors made byAnalog Devices. It will be appreciated by those skilled in the art thatother types of rotational sensors can be employed as rotational sensorsand that the ADXRS150 and ADXRS401 are purely used as an illustrativeexample. Unlike traditional gyroscopes, these rotational sensors useMEMS technology to provide a resonating mass which is attached to aframe so that it can resonate only along one direction. The resonatingmass is displaced when the body to which the sensor is affixed isrotated around the sensor's sensing axis. This displacement can bemeasured using the Coriolis acceleration effect to determine an angularvelocity associated with rotation along the sensing axis. If therotational sensors have a single sensing axis (as for example theADXRS150s), then they can be mounted such that their sensing axes arealigned with the rotations to be measured. One rotational sensor can bemounted such that its sensing axis is parallel to the y-axis and theother rotational sensor mounted such that its sensing axis is parallelto the z-axis. The operation and other details of the sensors aredescribed in U.S. Pat. No. 9,575,570, “3D Pointing Devices and Methods”,issued Feb. 21, 2017 (Liberty et al), and incorporated herein byreference.

In a stripped-down version of system 10 or should ball locator system 34and processor 38 malfunction, lose power, or a player simply opts to gowithout these components, remote 60 may be employed to manually controland direct retriever 12 in retrievals. Since players in many matches(especially outdoors) switch court sides every two games, system 10preferably includes a directional response capability in the sense thatretriever 12 is directed to the desired court side and not the other.Voice command can provide this as discussed, or T/R 43 (specifically,its receiver component that as discussed can be dedicated orstand-alone) as well as transmitter 62 can be configured to incorporatethis functionality. Since sensing a ball proximate to the net (asopposed to at a distance from the net) can result in increased falsepositives (the frequency of which also depends on the sensor types), adefault initial retriever 12 path initiated parallel to and alongsidethe net can decrease these incidences. Another approach is to employ anoverride via the signaling player remote 60, and then reactivatingsensor 42 to assume control of the retrieval after detecting the ball.After collection, retriever 12 moves toward the signaling remote 60 torelease the ball either as programmed or as signaled by the remote. Uponrelease, retriever 12 preferably returns to its standby positionautomatically via communication between T/R 43 (its transmitterfunction) and docking station T/R 69 (discussed further below) to directits path back. Alternatively, processor-controller 21 may includeprogramming instructions that upon execution calculate retriever 12'sposition relative to the court grid of the court in play along with itsstandby position, or at a minimum the latter in order to returnretriever 12 to its original standby position after a retrieval withoutundue additional manual control efforts.

System 10 preferably includes the option for a player/operator to selectmore than one retriever docking/standby position, via the GUI-typeselection, player remote, player control device, and voice command asdiscussed herein. For example, the operator can choose the directionalorientation of retriever 12 at standby; for example, three usefulsettings include a “neutral” position where the retriever 12 C/L alignswith the net's, along with two opposing orthogonal-to-net retriever C/Lstandby positions, with the position selection dependent on factors suchas the side of the net on which to prioritize retrievals, energyefficiency considerations, and retrieval response times.

In a retriever system configuration that employs a single playeremploying just one remote for retrievals alternating court sides alongwith that player (or should the players not alternate sidesperiodically, or per rule or convention every other game such as afteran odd number of total games played), system 10 may include a pair ofreceivers 63 positioned on opposite sides of retriever 12 or dockingstation 64. If operating on the same frequency, differentiating thedesired retriever 12 direction from the unselected side of the court viarelative signal power received could result in false positives due tonulling effects (caused by time path-interference or the like).Accordingly, the invention preferably includes a choice of receiver 63“handshake” routine selecting the court side for retrievals, e.g. bydeactivating the unselected receiver 63 or by way of processor-initiateddeactivating or nulling out its output. This also supports a two playerformat, a chair umpire (or other designated person or official,including any player designees or the like); for example, one or moreindicators displaying to each player the “call” status can be employedto either retrieve per that status or alternatively manually handover tothe other player remote or the like. Alternatively, employing a remote60 transmitter with 2 separate operating frequencies, one for each courtside/receiver 63, also eliminates the problem (as discussed below forvoice command functionality, receivers 63 may include an audio pickupcomponent, e.g. a microphone).

Another approach in singles tennis (2-player game) provides each playera remote 60 with a unique RFID or other means for system 10 todifferentiate between the two remotes. System 10 then includes means fordetermining which remote 60 controls a retrieval, as well as includingprogrammed instructions that upon execution direct retriever 12 towardthe designated court side (or for an off-court retrieval). System 10accordingly may be configured to identify which court side is associatedwith which remote 60 at any time, notwithstanding court changeovers.Remote and/or voice commands from a remote can then be convenientlyintegrated in the programming instructions with the commands for theselection of which court side is desired for a retrieval to avoidmisdirected retrievals. For employing just a single retriever for a2-player court, system 10 can further include monitoring the runningtotal of each player's power usage for their retrievals in the course ofa match, with each player's consumption updated and made available asdesired via data streaming to the player remotes or other means, e.g.configuring indicator 48 or 65 (described below) with this additionalcapability, or via periodic haptic or other alerts. A power budgetallocation, e.g. equal for each player or another ratio if desired, canbe selected as an input into the microprocessor, with programming tointerface with the player alert or other programmed-in functions, suchas a player cutoff upon reaching the allocation.

Remote 60 should be of a size and weight amenable for a player to carryit on their person during play, therefore its design should also factorinto these considerations. On-board proximity sensor 42 may be anultrasonic device or a video device, input to processor-controller 21and in the case of video input processed as discussed herein as withprocessor 38 processing of video input except that the processingoverhead may be decreased given the reduced ball recognition demands.Also, upon sensor 42 acquiring the ball location, an on-board indicator65 signals ball acquisition audibly or visually to alert the signalingplayer that manual remote control of retriever 12 converts to automaticsubject to user override. Alternatively, or additionally, sensor 42 maybe employed to sense other proximate objects, e.g. for obstacleavoidance and steering command control.

The invention optionally includes voice control and command technologyas manual control over retrieval actions or other functions in system10. This can be integrated with processor 38, remote 60, or combined inboth. Voice commands can serve as overrides and as user-selected optionsand for system setups or configurations and are much less intrusive anddistracting to adjacent court players when compared to the much louderand frequent score announcements during a match. Exemplary voicecommands include “Retrieve” and the like for directing retriever 12 tostart a retrieval, “Side A (or B)” followed by “1 (or 2)” and the liketo designate which [predesignated] court side and which court area toretrieve on, then “More” or “Less” or other suitabledirectional/steering commands to direct the immediate heading and pathof retriever 12 toward a ball until it's on course, as is furtherdiscussed below. Balls more proximate to the docking station/standbyretriever position can be approached at a slower velocity that moredistant retrievals, providing the operator adequate time to apply courseand heading correction instructions/signals, e.g. by voice command,until the ball proximity sensor “locks in” on a loose ball (upon whichthe retriever indicator light or a remote function indication). “Offcourt” and such when directing the retriever outside a programmed gridarea (e.g. outside the immediate court in play), “Collect [number ofballs]”, “Stop”, “Standby”, and other such commands to facilitateeffective and efficient retrievals. Commands can be incorporated intothe programming of remote 60 for implementation by remote controlcomponents such as a “TrackPoint” (a compact, isometric joystick, alsoknown as a pointing stick, style pointer, or nub), a GUI interface, amousepad, a conventional joystick, and the like.

In a preferred embodiment of the voice command-type system 10, the courtof play, the areas alongside it, and adjacent courts are assignedspecific alphanumeric and/or word designations convenient to voicecommand protocols. These commands may be selected so as not to soundlike common tennis parlance reserved for other communications, such asgame scores and other in-play verbal utterances (e.g. “let” or “letcourt”, “in”, “out”, and so forth), audible either from the match courtor from nearby courts. Court areas are also preferably differentiated byball retrieval zones designations, for example, the area betweenadjacent courts that includes retriever 12's standby location (applyinga command such as “Off” for a retrieval outside the lines on the sidewith the retriever, “Off-Far” indicating the opposite-side counterpart,and side “A” or “B” for the player's side on which to retrieve). Sincemost on-court retrievals will be in the service boxes on up to the net,and with each court side being a mirror image, subsequent commandinformation can designate an approximate location for either siderelative to a corresponding programmed-in court grid that may alsoinclude adjacent areas.

One grid according to the invention is to assign equally-spaced rowsdividing the service boxes from the net to the service lines andextending into the side court areas on both sides, for example, 5 rows.Each row is assigned a voice command callout, e.g. the numbers 1-5 fromthe net to the service box back line, the command “net” then “1-4”, or“net”, then “1-3”, then “line”. Columns add to the precision, but forease of use while maintaining sufficient precision in the retrieval path(taking into account the typical spatial and cognitive gridvisualization capabilities of the typical player/operator) anapproximate row distance callout should suffice. Additional voicecommands directing retriever 12 to “slow”, to turn (“left” or “right”),and then “straight” or “good” for the retriever to maintain that headingtoward the ball correct these approximations to obtain a successfulretrieve. Programming can also include automatically slowing theretriever as it approaches the command retrieve location until receivingadditional voice commands or a retriever sensor locks onto the ball toassume fully automatic control.

The programmed grid also preferably includes areas back of the describedforward court grid areas and can be distinguished using a command “back”or such along with a back row numbering system to direct retriever 12for back retrievals. An exemplary back court row designation is 1-3commencing from the service box back line, essentially mirroring thefront court grid configuration. The programmed retrieval grid mayinclude adjacent courts, e.g. “Next” for indicate an adjacent court,“Third” for another court away, and so forth. A command to “Hold” or“Stop” directs retriever 12 after retrieving a ball to delay the ballrelease; a command such as “And” directs retriever 12 after releasingthe first retrieved ball to await another command such as a second balllocation retrieve command instead of automatically returning to standby.Commands also can include “abort” or “standby” for retriever 12 toterminate a retrieve and return to standby. Some retrieves may beamenable to part or fully manual mode, at least until and after ballrelease (then auto return to standby whenever feasible or optimal).

Alternatively, a smartphone or similar device with an installedretriever app can serve some or all these functionalities, including theremote functions as well as the voice control/command functionality,which may be preferable in avoiding erroneous movement resulting fromextraneous noise sources. As discussed above, voice commandfunctionality may be integrated into the player/operator remote controlto communicate with one or more system 10 receivers 63 positioned on theretriever or docking station for controlling retriever 12 via the voicecommands. Alternatively, receivers 63 may include one or more audiopickups/microphones for receiving and signal-conversion ofumpire/non-player voice commands input to the processor which viaprogrammed instructions upon execution output retriever 12 controlsignals. The programming can include programming-in various predicativefactors, that is, desired play-cueing inputs, examples of which includeautomatic (and GUI-adjustable) end-of-point-play computations,play/serve resumption, and game status based on game score announcementsand the like, that may determine a retriever 12 standby status.

Commercially available voice command control interface software includesVRBot (Tigal Inc.), comprising a module with voice recognition functionsfor built-in Speaker Independent (SI) commands and up to 32 user-definedcommands (Speaker Dependent (SD) trigger or commands, along with voicepasswords (SV)). Other useful voice command/recognition technology isdescribed in U.S. Patent Application No. 20150095026, Bisani et al.,Apr. 2, 2015 (hereinafter “Bisani”), and incorporated herein byreference. Bisani describes an automatic speech recognition (ASR)processing system, in which the ASR processes speech based on multiplechannels of audio received from a beamformer. The ASR processing systemincludes a microphone array and the beamformer to output multiplechannels of audio such that each channel isolates audio in a particulardirection. The multichannel audio signals may include spokenutterances/speech from one or more speakers as well as undesired audio,such as noise from a household appliance. When receiving audio forspeech recognition processing, an ASR device may receive both desiredaudio, such as speech from a user, as well as undesired audio, such asaudio from other people talking to each other, noise from householdappliances, etc. In certain circumstances, the speech may be coming froma different direction than the noise. To isolate audio coming fromdifferent directions, a device may perform beamforming, a process ofisolating certain audio inputs using multiple audio input devices. Toperform beamforming, a device may include components for direction-basedprocessing, such as the microphone array and related components. Thedirectional based-processing components may be local to the microphonearray and/or may be separate but coupled to the microphone array. Themicrophone array may be fixed or steerable. When steerable, themicrophone array may allow for electronic focusing, where dataprocessing techniques may be employed to focus the microphone array onaudio coming from a particular direction, known as a beam. Such steeringmay be used to isolate the user's speech by focusing the ASR processingon audio received from the direction of the user. System 10's voicecommand subsystem preferably includes the described beam steering (giventhat the direction of audio commands from the player(s)/operator(s) isknown) when employing the microphone array. The audio from the selecteddirection is preferably processed to improve voice command recognitionand accuracy, such as adjusting signal-to-noise ratio depending on thevoice command direction. Other techniques such as reverberation, echocancellation, etc., may also be used to improve the reception of thedesired audio. The ASR device may simultaneously perform speechrecognition on the multi-channel audio to provide highly accurate speechrecognition results.

The identification of a voice command is preferably optimized bysimultaneously performing speech recognition in different directions toisolate the desired speech based on decoding multiple channels of audio.Receiver(s) 63 (or other suitable audio pickups as described) may alsoreceive interfering audio from different sources and directions, whichcan be identified and selectively eliminated from the voice commandsthat corresponds to the programmed-in system voice commands in order toselect that as the retriever 12 control commands. the speech recognitionresults/audio from that direction for further processing. Accordingly,system 10 may include a beamformer module to receive audio signals froma plurality of sources from different directions and simultaneouslyperform speech recognition in the different directions to isolate thedesired voice command. Of course, if voice command is local to theplayer/operator, e.g. built into the player remote or other personaldevice, then this more elaborate system is not required.

A 2-microphone array system is described in “Frequency DomainMulti-Channel Acoustic Modeling For Distant Speech Recognition”, WuMinhua, K. Kumatani, S. Sundaram, N. Ström, and B. Hoffmeister (IEEEInternational Conference on Acoustics, Speech and Signal Processing12-17 May 2019), and “Multi-Geometry Spatial Acoustic Modeling ForDistant Speech Recognition”, K. Kumatani, W. Minhua, S. Sundaram, N.Ström, and B. Hoffmeister (IEEE International Conference on Acoustics,Speech and Signal Processing May 2019), and is also within the scope ofthe present invention. System 10 may also include tracking meansattachable to one or more tennis players for communicating to theretriever a ball release location. On-board proximity sensor 42 for manyembodiments may be a less complex type sensor than sensor(s) 36 sincesensor 42 will typically not have to acquire (sense) a ball untilclose-by, given that sensor(s) 36 in cooperation with processor 38 andits vision program is capable of directing retriever 12 sufficientlyclose to a ball; consequently, less sensor (as well as less processing)sophistication is necessary. For example, a video sensor 36 andprocessor 38 (and/or processor-controller 21) programmed to employ naivecolor thresholding utilizing the ball color (optical yellow) istypically effective for optical yellow balls against varying courtbackground colors up to within close range (1-2 meters) of on-boardsensor 42 for hand-off. Sensor(s) 36 alternatively comprises anultrasonic range sensor, such as the LV-MaxSonar-EZ1 ultrasonic rangefinder.

As shown in FIG. 2, system 10 may include a retriever 12 docking station64 that can further include a means 66 for charging retriever 12's powersupply 22, e.g. a Li-ion, Ni-MH, or other type of rechargeable battery,and thus extend retriever 12's operational range and duration during amatch. Power supply 22 is selected based on the design's desired powerand useful operating time, as well as on the availability of in-userecharging options as noted. One such system is that often utilized byRC model cars, a rechargeable Ni-MH 4.8 V battery packs, readilyswitchable with standby fully charged replacement packs. Other powersupply 22 options include those utilized in electric scooters, electricbicycles, and motorized child ride-in vehicles, the rechargeable powersupplies for which vary in voltages from 12 V-48 V, and include varioustypes as already noted. Means for charging 66 may accordingly includeseveral different options, which may also depend on the intended use,e.g. an AC-outlet-stepdown transformer for an indoor court setting or ifoutdoors a solar cell and associated circuitry optionally incorporatedinto docking station 64. Retriever 12 can be programmed to return towhichever post is closest to its current ball returning location,although a preferred standby is in a docking station as discussed hereinwith a separate, connectable power supply for recharging the retrieveron-board power supply between retrievals while docked. An exemplarycharging interface is a conventional induction coil noncontact-typecharging system with the requisite complementary circuitry provided inthe two components (retriever and docking station). Tennis facilities(especially indoor) may opt to provide wired outlets to provide power.

The docking station also helps secure the retriever and itself can besecured to a net post; accordingly, either docking station 64 caninclude means 67 for securing/releasing to a net post, or in a morebasic configuration absent the docking station, retriever 12 can includethe means 67 for securing/releasing to a net post, e.g. to secure it inwindy conditions. For example, means 67 for securing/releasing retriever12 may be a magnetic coupling to attach/release to/from a net post 40;also, means 67 can be contoured (concave) to fit snuggly onto the roundnet post.

As shown in FIG. 7, an exemplary docking station 64 includes a base 100with a plate 102 rotatably mounted thereon and having a pair of grooves104 configured to guide and secure retriever 12 (via wheels 18) betweenretrievals in a retriever-standby position (indicated by the FOREarrow). Docking station 64 may also include a means 106 for rotatingplate 102 between positions (as illustrated by the dotted lines), e.g. astepper or other motor, or rotation can be effectuated by synchronizedopposing rotations of one or more wheels 18 within grooves 104.Structural detents 108 may be included to define/limit the degrees ofrotation of plate 102 (the net post 40 itself may serve in thiscapacity, in conjunction with other design aspects such as contact withblades 26). Alternatively, the amount of rotation can also be controlleddirectly via means 106 or system 10 can be programmed and/or manuallycontrolled to rotate plate 102 to orient retriever 12 in the desireddirection and into its orientation for redocking retriever 12 after ballrelease (where retriever 64 may travel in reverse and redock withoutunnecessary additional turning). Retriever location determining means 19in cooperation with proximity (or homing) sensors can facilitateredocking in accordance with system programming.

FIG. 8 illustrates a portable embodiment in which retriever 12 anddocking station 64 are attached/secured together on a rollable frame 300that comprises wheels 302, a telescoping handle 304, and means 67 forsecuring frame 300 to the net post 40. The configuration facilitatestransporting, setting up, and stowing system 10; it also provides anopen space in the middle in which to secure and hold a player's tennisequipment bag or other gear. Retriever 12 can be stowed in a folded-upposition (with blades 26 alongside frame 300) for compactness androtated into its working position (as depicted by the arrows) with frame300 secured to the net post 40.

As discussed, processor-controller 21 can be programmed to compute andtrack paths and distances to track retriever 12's ball-release locationand return it to the standby location. Docking station 64 preferablyincludes a T/R (transceiver) 69 to assist in docking maneuvers;alternatively, transceiver 44 can have this functionality.

Retriever 12 may have a sealed housing 15 for outdoor use in order toprotect its components from moisture and other elements, e.g. debris aswell as material from loose court clay surfaces, including rubico(Har-Tru), composed primarily of crushed metabasalt, or red clay(crushed brick or clay, crushed shale, stone, brick, or other unboundmineral aggregates). A sealed housing 15 also facilitates cleaning andhosing off retriever 12 after use. Since these surfaces comprise loosesurface material, it may be preferable to utilize for these applicationsa version of retriever 12 employing passive detents 50, 51, and othersdescribed herein, as opposed to the more complex active detents that maybe susceptible to diminishing performance and may also require morefrequent maintenance and repair due to the introduction and accumulationtherein of the soft court surface material.

Steering means 20 may comprise differential drive/differential wheel:the velocity difference between two motors drive the robot in anyrequired path and direction is “differential drive”; “differentialwheel” can employ two independently driven wheels fixed on a commonhorizontal axis. Another option is a “tricycle drive, employing threewheels with a fore steering wheel controlled by a motor and the two rearwheels attached to a common axle driven by a single motor with twodegrees of freedom (either forwards or reverse). Steering means 20 mayalso be an “Ackermann” design that mechanically coordinates the angle ofthe two front wheels fixed on a common pivotable steering axle with thetwo rear wheels on a second, fixed driving axle. Retriever 12 may alsocomprise continuous track (also called tank tread or caterpillar track)treads in lieu of wheels 18 to facilitate movement on loose tennis courtsurfaces, for which a robot tread-drive (“tank drive”) system may beutilized. Means for steering 20 then employs differential steering(movement is based on two separately driven wheels), applying more orless drive torque to one side of retriever 12 than the other to turn;this adds complexity and increases cost, requiring one or moreadditional motors and additional drive train components compared tonon-track drive systems. An additional consideration for the soft courtsurface is to avoid or minimize the possible effect of leaving tracks onthe surface from the retriever movements, accordingly, retriever 12 caninclude a means for smoothing out or otherwise grooming the track markssuch as a brush affixed to the rear of the housing or chassis and set ata level so as to smooth out the tracks while also minimizing the dragassociated with its contact with the surface. Other modifications may bedesired for grass court surfaces, such as an auxiliary ground releaseimpeller (described herein) and a ball support for collected balls (alsodescribed herein) due to uneven rolling on grass during retrievals.

Of course, the choice of materials for retriever 12 depend on severalconsiderations, including cost, weight, and overall utility in thevarious courts and conditions where it is used. The wheels and theirtraction are important to retriever 12's mobility and accelerationcapabilities, therefore the coefficient of friction between the wheelsand the various court surfaces is a factor. This may dictate alow-profile tire/wheel employing a neoprene or other such materialhaving a medium durometer and/or sufficient tackiness to enjoy a goodsurface grip. The weight of retriever 12 may also be a factor, asretriever 12's weight is a compromise between a heavier chassis aidingboth traction and stability against outside forces such as wind versus alower weight maximizing its acceleration, speed, and time-efficientcourt coverage.

Possible court conditions include rain interruptions as well as playduring windy conditions. As noted above regarding the former, retriever12 has a water-resistant design; regarding the latter, retriever 12'slateral surface areas are preferably minimized to reduce the effects ofwind, in particular cross-winds, on its retrievals so as to minimizeretrieval times and the occurrence of mis-retrievals and the like.Although housing 15 of course has a configuration and size in accordancewith system 10 and retriever 12's respective designs andfunctionalities, it preferably presents a profile, especially regardingits cross-sectional areas, minimizing the potential impact of windy orgusty conditions when used outdoors, and to which opposing blades 26 area sizeable contributor. It is therefore desirable to minimize theoverall cross-sectional surface area of opposing blades 26. Accordingly,opposing blades 26 may have multiple through-holes or slots 68 throughsidewalls 31 or comprise a loop with narrow walls and one large slot 68.Obviously, the clearance therethrough of each slot or through-hole 68must be smaller than the ball's diameter (provided above). Blades 26should extend at least 3 inches with respect to retriever 12's axial C/L(see FIG. 4) from housing 15 in order to define a collection space 30sized to retain a single ball; in multiple-ball versions blades 26extend with respect to the C/L at least those multiples thereof;accordingly, retriever 12 in some embodiments discussed herein may beconfigured to accommodate up to 6 balls or more (e.g. for tournaments orother situations utilizing same). Sidewalls 31 of blades 26 should havea vertical height dimension sufficient to retain/release a ball or ballsin/from space 30, which may depend on the embodiment. Thus, sidewalls 31are configured so as not to trap or hinder the ball for release butrather to position it on the ground at release with the blades thenguiding the ball toward the intended release point. For example, it maysuffice for an embodiment of blade 26 comprising passive detent 50 andupper lip 56 to have a vertical height dimension somewhere between theradius of the ball and its diameter (about 1.4 inches to 2.7 inches),although the height should be at least sufficient to accept the ballinto space 30 without it getting stuck prior to release.

Opposing blades 26 and housing 15 are either a one-piece constructionmounted on chassis 15 or blades 26 are a separate component(s) attachedto housing 15 (and/or chassis 14) via a fastening means 70. Fasteningmeans 70 in one embodiment is a latchable hinge allowing opposing blades26 to be rotated (as illustrated by the dotted lines in FIG. 4) betweena ball-collecting position and a folded-back, stowable position. Othersuitable fastening means 70 include a slotted (slide-in/slide-out-type)bracket or other such quick-connect; alternatively, blades 26 can beretractable blades 26 e.g. configured to slidably retractable withinguides along the top or sides of housing 15.

FIGS. 9-11 show an embodiment of blade 26 that includes a flared-out(leading) end 306 angled to deflect a ball into the collection space 30and broaden out the ball collection swath on both sides of retriever 12.Blade end 306 may be tapered as shown, the amount of which is limited soas not to allow a ball to slip over or under given a ball diameter of≈2.7″. Blade end 306 may also be twisted as shown (like a mobius striptwist) in order to present a leading edge surface 308 that is parallel(in the y-z plane) to retriever 12's to minimize end 306 snagging in thenet (the mesh or on the center tie-down strap) when the retriever is ita preferred initial retrieval path running parallel to and alongside thenet; this also lends to a preferred programmed instructions wherebyprocessor-controller 21 based on retriever movement input data detectsan unprogrammed stop or deceleration during a retrieval and directs acorrective maneuver such as a course reversal (subject to manualoverride). Blade end 306 can have a width (e.g. larger than 2 of themesh openings) sized to minimize catching/snagging in the net mesh.Passive detent 50 is concave area 51 extending upward from a lowerhorizontally-extending ball guide blade section 310 (vertically, atleast ⅛″-¼″ in height) that together with its opposing blade's ballguide 310 guides a ball at release from space 30 towards the target,while keeping it from riding back into area 51 and thereby maintainaccuracy at ball release. Each blade 26 preferably includes a wiringchannel 311, e.g. a longitudinally-extending groove (shown in phantom)on the underside of each blade connected via through-holes or ports tosensors or other components (e.g. LEDs 29) positioned on blades 26.

Blade 26 is shown with a slot 309 positioned aft of ball proximitysensor 33 for accommodating an additional, active detent 50 (e.g. arobotic claw as illustrated in FIG. 9) for selectively securing andreleasing a ball, providing retriever 12 with multi-ball retrievalfunctionality on a single retrieval “round trip”. Multi-ball retrievalcan also be provided by the other passive and active detents describedherein, the choice of which depends on the desired system 10 design.

Blade-mounted ball sensor 32 upon detecting balls' movements in and outof space 30 signals inputs into processor-controller 21 for controllingretriever 12 maneuvers and ball retrievals as discussed. For example,upon collecting a ball, retriever 12 can either commence a ball releaseoperation or another ball collection operation as directed, and uponcompletion commence the next programmed or directed operation, usuallyending with its return to standby and/or docking station 64. A proximitysensor positioned on top of the retriever or on each blade can helpmaintain a desired distance on retriever approaches alongside the net,e.g. to maintain a programmed distance away from the net. Alternatively,a sensor centrally-positioned and mounted above the blades, e.g. on thefront roof of the retriever, at a specified range detects the ball anddirects the retriever toward it but so as not to override the bladesensor protocol but to apply an adjustment to close in on the ball uponit being picked up by the sensor at a programmed-in delta distance gapas directed by the central proximity sensor.

The number of such proximity sensors may be minimized or consolidated byconfiguring system 10 to utilize a sensor so as to“recognize-differentiate” a sensed, proximate object based on itspresent location, position, direction, and/or orientation of retriever12 via programmed instructions and the programmed-in court grid. Forexample, upon the initiation of a retrieval, system 10 can be programmedas discussed herein to commence alongside the net, with a proximitysensor utilized to ensure a desired distance maintained betweenretriever 12 and the net until it's on a maintainable path sansproximity sensor feedback, upon which the proximity sensor output can beutilized to indicate a proximate ball or player.

FIGS. 12-16 show embodiments of retriever 12 capable of collecting abouncing or deflected ball, with or without a ball ejector 52. The aftend of housing 15 includes interior sidewalls 313 with surfacescontoured (e.g. concave) to deflect and contain a ball on the bounce. Asdiscussed above, retriever 12 may also include the ball ejector 52 alongwith a slightly concave ball support 54 (or a through hole, e.g. a ¼″-¾″hole) for collecting and passively securing a bouncing or deflected ballthereon. A ball proximity sensor 32 may be positioned in or under one ormore of the ball support 54 through holes, e.g. as discussed below inthe embodiment illustrated in FIG. 18). Retriever 12 can include a roof312 to augment the detent capability of concave damping areas 51 (FIGS.9-10) and deflect balls forward and into collection space 30. Roof 312can also include a concave (as viewed from above) channel 314 with afore-to-aft decline (negative slope) to funnel balls aft ward (eithertoward ball support 54/ejector 52 or sans this component into space 30for ground release). Sidewalls 313 and other ball-impacting surfaces canoptionally include a resilient/impact-absorbent surface treatment (e.g.texturing or ribbing) or a top layer, such as rubber, a syntheticelastomer (neoprene or urethane),or foam or the like, to dampen andabsorb ball impacts and maximize capture-retention.

Ball ejector 52 may be a solenoid 400 (preferably a linear type)operating a core piston-striker 402 having a ball-striking surface 404for striking the ball for an ejection release, such as in US PatentApplication No. 2015/0061224A1, filed Aug. 7, 2014 (Ron et al), andincorporated herein by reference, that describes a solenoid strikerassembly comprising a spring-biased striker member activated by asolenoid, where the solenoid moves the striker against the spring into acompressed position, allowing the spring to snap the striker against theball upon solenoid de-energization/striker release.

Retriever 12 may also include a ball queuer/feeder 406 with a front end407 and a back end 408 having a fore-to-aft decline (negative slope) forgravity-feeding a ball back toward ejector 52. End portion 407 (fore)has a leading lip 412 sized to receive and hold a ball on surface 413that is configured (e.g. slightly concave) to releasably secure theball. Lip 412 is pivotally attached to aft end 408 via a hinge 414 orother pivot means, raised and lowered via an operator 416 (e.g. amotor-gear assembly, a solenoid or the like), whereby with lip 412lowered a ball received thereon is lifted up and gravity-fed down ballramp 410. A ball proximity sensor 32 as described above senses a ballrolling onto lip 412 upon which processor-controller 21 initiates thefeed-queue sequence, in conjunction with a means 418 for controlling thesequential feed of a ball or balls to ejector 52 such assolenoid-operated detents (or a robotic claw gripper as illustrated inFIG. 9, or back-to-back such grippers with a ball pass-through feedingcapability) positioned at a height so as to secure the ball near or atits great circumference (≈1.30″-1.40″); another sensor 32 adjacent ballramp 410 senses a ball fed back toward ejector 52. In lieu of asolenoid-piston, a spring-lever arm ejector or other ejectors 52 knownin the art are also within the scope of the invention. Front end 407optionally includes a ball ejector 420 (solenoid-piston type or thelike) positioned for striking and ejecting a ball horizontally/forwardfor a ground release as an alternative to (or supplementary in) inertialground ball releases in some embodiments, e.g. the combinationejector-inertial (ground release) embodiment described further below. Inthese embodiments, retriever 12 can be configured to rely on raising uplip 412 to secure a ball pending release, to substitute for any or allthe passive and active detents (e.g. concave areas 51 on blades 26).

Ejector 52 optionally may include a ball launch guide 500 positionedadjacent to and extending above ball ejector 52 for directing the ballupward in a desired release direction. Guide 500 preferably includesadjustment means 502 for adjusting the launch angle and ejected balldirection. Adjustment means 502 includes guide-urging means 504 (e.g. asolenoid-operated piston assembly or a geared worm-screw assembly)secured at one end to housing 15 and at the other end to guide 500.Guide 500 at end 507 is pivotally mounted to retriever 12 by pivot means508 (e.g. a hinge assembly as shown). Guide 500 is then adjustablemanually or via programmed instructions to guide a ball in a desireddirection and trajectory. Ejector 52 can also include an adjustableforce functionality to control the ball release exit velocity. Optionalobject proximity sensor(s) 509 can provide input for adjusting theseparameters as desired for optimal performance including safetyconsiderations (e.g. detecting a player's distance therefrom to ejectthe ball safely), including programmed or manual retrieval interrupts.System 10 can further include an override function to allow a player orindependent operator, e.g. an official such as the chair umpire, tooverride at any time the proximity interlock in order to eject a ball orballs as desired in each circumstance.

In the embodiment illustrated in FIGS. 15-16, retriever 12 is configuredto collect both bouncing and rolling balls and can release a ball eitheron the ground or in the air. Retriever 12 includes the ballqueuer/feeder 406 for directing balls from space 30, either aft bygravity-feed on ball ramp 410 into ball support 54 for ejection, orforward for ground release. In this embodiment, ball support 54 includesa ball support rotating means 600 consisting of a plate and a steppermotor or the like rotatably mounted on ball ejector 52, with rotationdirected by processor-controller 21 and its programming/inputs. Guide500 has a lower housing 602 secured to ball support rotating means 600that is either (a) integral with guide 500 (such that rotating ballsupport rotating means 600 and guide 500 occur together), or (b) is aseparate assembly (just lower housing 602 rotates with ball supportrotating means 600 and not guide 500). Lower housing 602 includes a ballfeed-through opening 604 at the bottom of housing 602 sized for passagetherethrough of a ball (i.e. >2.7 inches diameter for a tennis ball),where opening 604 is preferably arched as shown) between ramp 410 andsupport 54. Rotation of ball support rotating means 600 moves opening604 away from ramp 410 to position a wall 606 of housing 602 in linewith ramp 410, to function as a detent (along with additional detents asare described herein if desired) blocking ball-feeds aft until space 34is available for a desired ball feed. Ball support rotating means 600 isthen rotatable back into position to recommence ball feeding aft. It isalso noted that balls can be fed forward of space 34 in some embodimentsdescribed herein, e.g. in two-way ball feed versions. Of course, theexact timing and sequencing of ball feeds and ejections is subject toprogramming or manual direction as desired for a given application. Theoutboard surfaces of housing 15 and blades 26 preferably align along acommon vertical (x-z) plane to optimize ball collections close to thenet, e.g. when retriever 12 travels alongside and parallel to the net ineither direction.

Another potentially difficult retrieval situation is when a ball is inor near a corner of the court (i.e. behind the baseline). In order toalleviate this retrieval issue, the invention includes means (notillustrated) to block or deflect balls from the corner, e.g. atriangular-shaped block or a strap (e.g. a bungee-style cord) affixedonto the fencing on both sides of the corner to block off the corner anddeflect the balls. Alternatively, retriever 12 may include a means forgrasping and securing a ball as is further described below.

FIG. 17 is a logic flow diagram exemplifying sequences for carrying outvarious embodiments of the invention. The diagram is self-explanatory inthat it incorporates much of the logic, process, and programmingdescribed above to consolidate and convey the utility, novelty, anduniqueness of the invention in its multifaceted, numerous embodiments.It is further noted that the subroutine “Preset Court Grid Dimensions”includes as possible programmed (and, of course, user-modifiableparameters) the different racket sports' courts dimensions enumeratedherein, as well as the options for indoor versus outdoor layouts andconfigurations, differing sizes (e.g. on larger or smaller areas behindbaselines), and off-court (i.e. off the actual court in play) desiredretrieval expansions, e.g. when adjacent courts are vacant andretrievals are expandable. A voice command protocol may include commandsdifferentiating between a retrieval off-court immediately adjacent tothe court of play, other adjacent or distant courts (e.g. using acommand “multi-court” or similar), as well as fence retrievals calledout as such, as further discussed below. Much of the logic illustratedin the figure is programmable into processor-controller 21 as discussedabove. Variations are also within the scope of the invention asdiscussed elsewhere herein; e.g. the “Retriever Location Calculation”preferably includes applying a correction factor as necessary not justfor computing a return path to the standby location but also forcomputing the retriever path to the ball release point or for furtherball retrievals.

System 10 can accordingly include grid selection options and programmingfor player(s)/operators extending the collection grid on adjacent courtsor spaces, with optional indicator lights 48 and 69 configured todisplay the status (e.g. “on-court” vs “extended”). If desired, anotherproximity sensor 42 can be mounted on net post 40 for sensing playeractivity in the extended grid areas for reasons of player safety to thenautomatically limit the range of retriever 12 (and where the adjustedstatus can be indicated as such).

Of course, there are sources of error inherent in some facets of system10 that could degrade system performance, and the “Retriever LocationCalculation”. Retriever 12 during a retrieval may be commanded totraverse a number of course headings and distances, with multiplelocation calculations and thus a larger cumulative error.Processor-controller 21 can include standard path error correctionalgorithms (least squares, smoothing, and the like), while system 10 caninclude return path programming that directs retriever 12 (preferably inreverse as noted above) toward the sideline on the intended docking post40 location, and the path toward the sideline can be error-correcteduntil the docking handshake is established between retriever 12 anddocking station 64. Another docking routine described below utilizes asensor array 810 on the back end of retriever 12 responsive to a homingor other linking signal emitted from or bounced off of docking station64.

A useful retriever location calculation correction technique utilizesoptically—(or other sensor—) readable fiducial markings such as courtlines for updating the probable retriever location relative to theprogrammed court grid layout. One such technique calculates the angle atwhich retriever 12 crosses a line and compares the actual angle to apredicted angle based on the original path, and updates accordingly. Asingle-sensor technique is described in “Robot Odometry Correction UsingGrid Lines on the Floor”, P. Machler, Swiss Federal Institute ofTechnology, Lausanne, Switzerland, MCPA 1997 Pisa, Italy (hereinafter“Machler”). Machler describes a robot mounted with a single,downward-oriented light sensor capable of identifying a painted line ona floor surface, determining the angle of the robot crossing the line,and applying a correction factor to the odometry based on the anglecalculation. The present invention includes this technique, as well asthe computationally-simpler technique of positioning a sensor mutuallyadjacent on each side of retriever 12 and, based on the timedifferential between the sensors in sensing a line, calculating theretriever angle to the line to determine the angle and then the locationcorrection factor. The light sensor provides an ideal short pulse whencrossing a grid line that is used to synchronize the supposed positionof the robot (calculated by odometry) with the real position.

Another probable retriever location computation technique relies on themicroprocessor's internal clock in coordination with precision speedcontrol and programmed-in path travel to guide retriever 12 from itsstandby location to a specific grid row or other designated pathlocation. The programming includes retriever steering commands based onthe time-speed-orientation computation(s), with travel commencing alongand turns in accordance with the command path. Programming in standardinitial retrieval paths to follow to specific locations as directedthere by the operator voice command (estimated grid ball location asdiscussed above) along with operator retriever steering commandadjustments (e.g. via voice command (“right”, “left”), manual control,WII-type device, and so forth), coordinated with specified sensorinputs, is a useful, non-GPS-based local navigation system. Thecalculations are also useful for directing the ball releasedirection/velocity vector and release location, as well as the return tostandby path routine. An exemplary return path is dictated by readingthe magnetometer (orientation of retriever), reorienting to a desiredinitial return path orientation, and then reading the traversal of courtfeatures/fiducials such as the court lines to compute the desired returnpath. Once docked, it may be preferable to forego rotating plate 102 toits neutral, unrotated standby position and instead await the nextretrieval, which may be on the same court side, in order to conservepower if a factor and extend the retriever's availability during amatch.

FIG. 18 illustrates another embodiment of the invention capable ofreleasably securing multiple balls, employing a ball tray 700,optionally including a slanted-down leading lip 702, positioned betweenor integral with the bottom portions of blades 26 and extendingfore-to-aft as shown and so as to allow sufficient ground clearance forthe front ball collection end when weighted down by the collected ballsand the assembly itself. Tray 700 includes multiple spaced-apart ballsupport surfaces 54, as shown for up to three balls, that being thetypical number in play for most recreational and competitive matches. Aball proximity sensor 32 is positioned to sense each collected ball,e.g. within or under ball support surface 54 as discussed above ormounted thereabove on the underside of roof 312 (see FIG. 14). Activedetent means 418 (a solenoid-operated detent as shown) is positionedfore of each surface 54 for releasably securing a ball until release.Wall 704 just aft of the rearmost ball support surface 54 for securing aball thereon may further include a ball ejector 512 (see FIG. 14) forstrike-ejecting the ball(s) at the programmed release point. Sensors 32and detent means 418 can be actuated in cooperation with user-inputcommands directing a retrieval for a designated number of balls. Forexample, for a one-ball retrieval, the intermediately-positioned detentmeans 418 can be actuated to serve as a backstop, and when the foremostsensor 32 is triggered, the foremost detent means 418 is actuated totrap the one ball until release. The process works similarly for the twoand three ball retrievals. Release can then be conducted in a controlledmanner toward the target. For example, with three balls collected,retriever 12 accelerates to a first release point where the foremostdetent means are retracted, retriever 12 decelerates, and the first ballrolls toward the target. After a programmed interval, retriever 12accelerates again to a second release point, and so forth. The processorprogramming can include instructions that upon execution sequentiallyrelease consecutive balls in this manner, so as to space apart eachground release and roll each ball toward the target with sufficientvelocity to allow a player sufficient time to receive and collect eachretrieved ball. The programming may thus encompass the desired intervalsand retriever 12 acceleration, deceleration, and speeds determined bythe release locations' distances from the target, to a programmed finalrelease velocity for each release and then the deceleration or full stopto impart sufficient inertial velocity to the ball to release from thetray and roll to the target.

Referring now to FIG. 19, tray 700 may include through holes or slots706 for allowing court debris or soft court surface materials to settleback on to the court and clear tray 700 of accumulation, particularlyfor clay court operation. Retriever 12 may further include means 708 forclearing tray 700 of court surface debris, e.g. a blower-fan unit orforced air jets positioned behind and/or alongside tray 700 asillustrated (or, brushes or other mechanical removal means or the like).Soft courts with raised court lines/nails and hard courts having cracksor uneven repairs present less-than-ideal operating environments,therefore, retriever 12 may further include a means 710 for raising tray700 to surmount the surface irregularities and thus minimize retrievaldisruptions. As illustrated, means 710 comprises one or more opposingslot pairs, where the forward-positioned opposing slots include a curvedportion permitting tray 700 to lift up and back when encountering araised court area and with tray 700 in the unlocked position. Theunderside of tray 700 may further include one or more skids (notillustrated) to prevent tray 700 from catching on raised lines/nails bybumping tray 700 upwards. Tray 700 can also be locked at the front slotpair if desired when running on a smooth court surface.

In lieu of placing corner deflectors (described above) to eliminatehard-to-retrieve corner fencing-lodged balls or to facilitate otherdifficult retrieves, retriever 12 may include a means 800 (see FIG. 19)for grasping and securing a ball for otherwise difficult-to-retrieveballs, e.g. in or near a corner, up against perimeter fencing,obstructions (e.g. an equipment bag, a chair support, or the like), orother tight spaces. Means 800 as illustrated comprises an arm 802, e.g.an elongated curved member, half-robotic claw, or the like, pivotallymounted above collection space 30 and between the tips of blades 26, andalso a means 804 for moving arm 802, e.g. a stepper motor,solenoid-piston, or the like, from a raised, nondeployed position to alowered, ball-grasping-securing deployed position as indicated inphantom. Means 800 may be activated and retracted in coordination withone or more proximity sensors 806 positioned on blades 26 or elsewhereon retriever 12 (e.g. at/on its fore end, e.g. on the leading edge ofarm 802 as shown, to the side on the leading edge, or on top of roof312), whereby upon detecting proximity to the fence or obstruction,sensor(s) 806 outputs a signal/indication to an input of processor 21that in response executes programmed instructions directing means 804 tomove arm 802 into position to grasp and secure a ball. Subsequently uponother processor inputs described herein signaling a completedcollection, processor 31 directs means 804 to move arm 802 back to itsraised, standby position.

In a basic configuration with blades 26 fixed (that is, notretractable), and with arm 802 positioned to make contact with a ball,the ball is accessible even with retriever 12 up against an obstructionor corner fence portion. In the latter case, retriever 12 can approachthe ball alongside a run of fencing rather than at an angle directlytoward the corner which could interfere with arm 802 grasping the balldue to fencing interfering with blades 26. For these retrievals, themeans for determining retriever position/location can interface with theprogrammed-in court grid/dimensions and side-clearance object proximitysensors to compute when to activate means 800.

A more direct, “heads on” corner ball retrieve, i.e. with retriever 26approaching the corner ball along the corner diagonal, is feasible withretractable blades 26, although this obviously adds complexity to theoverall design. Another approach to this retrieve is to utilize aprotruding (or retractable/extendible) arm 802 that extends sufficientlyforward of blade ends 306 to contact and grasp the ball despite fencecontact with blade ends 306. Means 800 can be activated either manuallyor by a programmed-in voice command, e.g. “fence”, “corner”, or thelike. The retrieve path protocol can include retriever 12 upon securingthe ball automatically reversing from the corner/obstruction, with arm802 lifting only upon the ball being secured for later release.

Referring again to FIG. 19, retriever 12 in addition to (or in lieu of)indicators 48 or 65 may include a message screen or indicator array 808on each side to display an indication of the information to theoperator, such as the present status of a retrieval in traversing acommanded course. The readout in this instance could comprise a statusbar increasing (or if desired the reverse) in length to enable theoperator to gauge the retrieval progress and adjust it as needed withfurther voice commands or other input.

The embodiments of the invention discussed above apply mainly torecreational play rather than professional or other tournament-formatsettings that utilize ball-persons for loose ball retrievals. Theinvention is also useful for the latter where it can speed up andexpedite play, an issue of increasing importance. USTA rules now requireplayers to adhere to specified time restrictions and tournaments provideclocks to alert them to the countdowns. The clock is turned off duringpoints, but it rarely rests when the ball is not in play. The countdownstarts the moment players walk on court, counting down the one minutethey are allowed between arriving at their chairs and attending thepre-match coin toss. The clock then counts down five minutes for thewarm-up, and one minute again between the end of the warm-up and thestart of the match. Players are given 25 seconds to start their firstserve, and between games, it counts down from 30 seconds; duringchangeovers, it counts down 1 minute 30 seconds; and between sets, twominutes. It also displays the three minutes allotted for medicaltimeouts. The first time the clock hits zero before a player begins theservice motion, the player receives a warning. For every subsequent timeit happens, the player loses a first serve. The returner is instructedto play at the pace set by the server. Speed of play therefore hasliterally become “of the essence”. Pro tournaments also generallyutilize thrown and rolled ball techniques during retrievals.

Retriever 12 may therefore include both capabilities, operationallyselectable by either the operator (umpire, ball-person, or player) or bysystem 10. Accordingly, ejector 52 and ball queuer/feeder 406 cancooperate as a two-way feed path (shown by the double-headed arrow inFIG. 14). Roof 312 preferably has a fore-to-aft decline (negative slope)for directing bouncing balls into aft collection space 510. Ball ejector52 may include an adjustable force capability that permits the operatoror system to engage in 2-way ball feed either toward or out of theejector and toward the front for inertial ground release, oralternatively, an additional ball ejector 512 (e.g. a solenoid-pistontype) positioned orthogonal to ejector 52 can provide this dualcapability to eject a ball either up or forward. A second concave ballchannel 514 on the aft forward-facing upper housing 15 may be includedto funnel balls into aft collection space 510 through the opening behind(aft of) roof 312. The forward-positioned ball proximity sensor 32 canbe positioned on the underside of roof 312 in lieu of on a blade 26.Retriever 12 in this configuration may further accommodate up to 6 ballsor more (e.g. for tournaments or other situations utilizing same) by theappropriate sizing and relative placement of these ball-feed components,and can optionally include an additional feed bin (not illustrated)connected to same.

It was noted above that an advantage of the invention is in professionaltournaments that use ball-persons for retrievals. Given the presentSARS-Covi-2 (“Covid-19”) pandemic and the transmissibility of virusesvia contact with surfaces, the less handling of the ball, the safer itis for the participants. System 10 or retriever 12 optionally includes agerm-killing ultraviolet module such as that described in U.S. Pat. No.9,212,067, “Water Bottle with Flow Meter”, Gellibolian et al., issuedDec. 15, 2015 and incorporated herein by reference. The uv module can bepositioned in a ball feed chamber or other suitable ball receptacle, andinclude an interior surface coated with a reflective material tooptimize the uv energy generated by the uv light emitting diodes (LED)hitting the ball surface, as described in Gellibolian et al.

Professionals in tennis tournaments as well as other tennis players havediffering preferences concerning selecting which ball to put into play.For example, system 10 can be programmed to respond to the operator orplayers' selected preferences or respond via voice command or remotecontrol to provide a ball or balls on command. A player can then chooseto have retriever 12 provide a varied combinations of retrievalpreferences, whether it's a mode of ball release—rolling or bouncing theball (e.g. to be caught on the racket face) or launching (to be caughtby hand), or a combination of these, as well as including the option toreject a ball before serving and signal for another, and so forth. Theinvention can further include player-selected options such as the ballrelease timing interval, ball release velocity, trajectory and the likefor rolling or ejection releases. The fiducials or alternativeball-queuing-targeting means discussed herein can facilitate and helpenable this capability. Fiducials may also be employed in other aspectsof the invention. For example, a fiducial can be assigned to the dockingstation and/or net post, and programmed to interface into the docking,redocking, and other functionalities incorporated into system 10involving the docking station.

Another version (not illustrated) utilizes a compressed gas (pneumatic)system to apply the motive force to the ball to eject via the chute or aground-release; it may also be configured to work from a single pressuresource using check valves or the like to direct the motive force for thedesired ejection type.

System 10 may further include programming instructions that uponexecution cause ball ejector 52 in combination with a vectoring ofretriever 12 to direct balls by ejection and/or rolling in a desireddirection and at a desired velocity, for example toward players in amatch sans ball-persons, or in other scenarios, e.g. in a high leveltournament setting, where system 10 cooperates with tracked or setlocations of ball-persons or players, and optionally also with aball-tracking system such as described above, with retriever 12 beingdirected to return a ball in accordance therewith.

The retriever system processor in combination with the ball trackerpreferably includes a ball intercept algorithm the instructions of whichupon execution compute an optimized retriever-ball intercept time,location, and/or path and direct the retriever with respect to speed andangle of approach to optimally intercept and collect a bouncing ball.The retriever system preferably includes an abort routine as discussedabove and may further comprise override capabilities under the controlof the operator remotes for directing the release of the ball(s) in anoptimized direction and to an optimized location. For example, insteadof rolling the ball(s) back toward the initiating remote/operator, bymeans of an override the ball(s) may be released toward an alternateoperator and/or location on the court, such as toward the baselineareas. Further control may be assumed, e.g. for player safety purposes,to release the ball(s) as commanded by a remote in the direction of thatremote. This would allow a ball-person to direct balls away from aplayer's path.

Professional tournaments utilizing ball-persons by also utilizing theretriever can optimize the combination by decreasing the number ofrequired ball-persons. For example, tournaments often position twoball-persons at the back of each court side as well as at least one andusually two near the net, one by each net post. One or both ball-personsby the net posts can be replaced by one or more retrievers that canretrieve balls in or near those areas and then return the balls in thedirection of the ball-persons behind the baselines. For example, evenreplacing just one ball-person by the net posts decreases the staffingrequirements and facilitates that ball-person's job in retrieving netballs toward the other sideline. Of course, the alternating of servesfrom one court side and to the other more often than not results in“lets” (balls hitting the top of the net and dribbling over), leading toretrieval and match delays minimized by substituting the retriever fornet-caused or net-side retrievals. Accordingly, employing the retrievernot only speeds up play but also decreases the distraction to theplayers since the retriever is smaller and less visible (for example asthe players are reassuming their service and receiver statuses andball-persons are not in their direct or peripheral lines of view).

The retriever system enables a ball-person by a net post (or absent thembehind the baseline(s)) by means of a player/operator remote to controlretrievals as needed. Accordingly, one or more retrievers can bepositioned outside the net posts or at other positions around the tenniscourt to optimize the retrievals. If a retrieval is commenced but theball travels in an unpredicted direction, e.g. landing on the oppositeside of the court from a commenced retrieval, the system particularly inthe pro configuration preferably includes programed instructions thatupon execution respond to an operator's remote control signal or voicecommand to abort a retrieval and/or actuate a retrieval by an alternatesecond retriever. This command is also useful should the retriever forwhatever reason go toward the wrong court, and the ‘abort’ command orthe like can be followed by another command redirecting the retriever.The retriever system of the invention may also be used withcommercially-available ball tracking systems/algorithms, such as theHawkEye® ball tracker or others that have both location and 3-D trackingcapabilities to thereby provide 3-D data for retrieving a bouncing ball.The data is processed as described with the retriever controlled anddirected as described.

Accordingly, system 10 in another embodiment expedites play by employingone or more retrievers 12 in conjunction with one or more remote controldevices selectively controlled by one or more ball-persons to assist inretrievals. System 12 preferably includes programmed instructions thatupon execution calculate an optimized retriever-assisted retrievals ofloose balls (e.g. based on parameters such as the ball location relativeto the locations of the remote control devices/ball-persons), therebyselecting the retriever 12 providing the optimal retrieval efficiencyfor retrieval and release/return over other retrievers or over a manualretrieval option. Possible standby locations for retrievers 12 can beselected based on the overall court dimensions including off-courtareas, and where the chair ump, ball-persons and line judges arenormally positioned, and so as not to present a safety hazard to theplayers and other participants. One safe, central location is under thechair umpire's chair.

Referring again to FIG. 19, retriever 12 optionally includes a sensorarray 810 preferably positioned on its aft housing responsive to ahoming signal such as an optical beam or RF signal emitted from dockingstation 64, e.g. from T/R 69 or a separate dedicated signal emitter. Thedocking maneuver is accomplished by adjusting steering commands asnecessary to align array 810 with the emitted signal and thus adjust theorientation of retriever 12 as it approaches station 64 to ensure asuccessful docking.

The docking station embodiments described and shown herein areillustrative in that other designs not illustrated are considered withinthe scope of the invention. For example, the turntable-typeconfiguration described above with means 106 rotating plate 102 cancomprise an array of selectively-activated electromagnets that incooperation with a corresponding array positioned on the rear housing ofretriever 12 can position the retriever and/or turntable to a programmedor selected position. Another docking system (not illustrated) utilizescourt-installed fittings configured to receive corresponding retriever12 couplings which may include hard-wired charging means and othercomponents described herein. One location is in the area adjacent a netpost, and under an umpire's chair or its vicinity so as not to present afurther obstacle to players during match play.

Other embodiments are also within the scope of the present invention andit is not limited thereby. For example, retriever 12 for steering means20 may employ a swerve/crab configuration, where wheels modules rotateon the vertical axis to control direction, and which typically employs 4traction wheels, with a representative range of retriever speed of about8-12 ft./s. Omni-directional retrievers have good maneuverability andefficiency at the expense of some extra complexity. Suitable drivemotor(s) 16 may comprise 3 or more motorizing wheels (e.g. Mecanumwheels) to provide independent tangential, normal and angularvelocities. Three wheeled systems are mechanically simpler, but fourwheels provides better acceleration using the same motors along withimproved traction, such as described in is “Dynamical Models forOmni-directional Robots with 3 and 4 Wheels”,https://www.researchgate.net/publication/256089847_Dynamical_Models_for_Omni-directional_Robots_with_3_and_4_Wheels.“Fast and Precise Positioning of Wheeled Omni-directional Robot withInput Delay using Model-based Predictive Control”, M. Zarghami et al.,IEEE Proceedings of the 33rd Chinese Control Conference, ISBN:978-9-8815-6387-3 (28-30 Jul. 2014), describing the trajectory trackingof an omni-directional mobile robot by model predictive control (“MPC”)in the presence of input-delay, where the MPC controller is employed tothe end of trajectory to achieve a significant reduction in the time ofrobot travel. According to the authors, the approach achieves about a 40percent enhancement over a PID controller in time to reach target andresults in improved robot control in target point error.

Omni directional robots are built using omni wheels and/or castors.Since Omni wheels have smaller wheels attached perpendicular to thecircumference of another bigger wheel, they allow wheels to move in anydirection instantly. The major advantage is that they do not need torotate or turn to move in any direction unlike other designs. In otherwords, they are Holonomic robots and can move in any direction withoutchanging the orientation.

EXAMPLE 1

A retriever 12 was built by modifying an off-the-shelf Simrex A130 1/16scale 2-wheel drive RC remote control off-road truck, with a ballcollector 24 mounted thereon including opposing blades 26. The retrieverwas tested on a hard-court surface, successfully collecting balls at thenet as well as in areas away from the net and releasing the balls towardthe center service-mark on the baseline. A number of different bladeconfigurations were thus tested in order to define and refine bladeparameters such as length, contours, optimal sensor placements, andpassive detent configuration and the like, to finalize the designparameters for the 3D-printed prototype blades (“test blades”).

The test blades 26 were fabricated as mirror-image structures of oneanother using an AutoCAD-programmed 3D-printing process employingacrylonitrile butadiene styrene (ABS) polymer, to the followingspecifications:

Description/Drawing Ref. Num. Description (“L” = length, “W” = width,“D” = depth) Detent 50 Concave middle section: 7″ L (at top); 2⅞″ W(from bottom ⅛″ ball guide); 1½″ D_(max)″ at top center Blade 26 Blade:16″ L; 3″ W; ⅛″ D Blade end 306 Blade flare : 26° angle from blade ballguide (310); 3″ L; 2″ W_(min) (at leading fore edge)

For testing, blades 26 were spaced apart 3″-4″, secured aft to a robotplatform, and by a pair of spars above the collection space with thefore spar just aft the blade flares and the aft spar just aft of dampingareas 50. The fore spar emulated a leading edge of a roof 312 to testits functionality for redirecting and containing a ball deflected aftand bouncing off the flares in high speed ball retrievals. Previously,testing had confirmed the equivalence between a moving retriever ballretrieval and stationary retriever tests (where balls of a knownvelocity (ramp-accelerated) were directed against the blades), thelatter while video-recording from just above the blades to study thecollection dynamics frame-by-frame, as follows.

EXAMPLE 2

The operating characteristics of the test blades were first studied bysecuring the blades in a) a stationary, fixed position and b) arotatable position, and directing tennis balls at and into the testblades at varying angles and varying ball velocities, whilevideo-recording the tests and studying the test blade performanceframe-by-frame. Balls were fed through a 3″ ID 10′ length PVC pipefitted within a 4″ ID 10′ length PVC pipe to provide both flexibility invarying ball velocity and to permit balls (≈2.7″ diameter) to freelyexit the angled 4″ pipe opening. The pipes were set at differing heightsat the feed end to produce varying ball exit velocities, from 3 ft/s to10 ft/s.

-   -   (a) Stationary tests: tennis balls at higher velocities were        captured effectively notwithstanding some bouncing off the fore        spar. Lower velocity retrievals were routine.    -   (b) Rotating tests: the retriever was rotated upon a ball        rolling past the fore sensor 32 slots at both higher and lower        speeds, and the balls stayed in the collection space 30 and were        observed to utilize both blade passive detents 50 (concave        areas) that functioned as damping means in retaining the balls.

The dimensions of retriever 12's ball collector 24 are based on theofficial tennis ball dimensions established by the International TennisFederation: diameter 6.54-6.86 cm (2.57-2.70 inches), and a mass of56.0-59.4 g (1.98-2.10 ounces). Accordingly, ball collection space 30for tennis must have a minimum width of at least 2.70 inches fore-to-aftback to or near wall 28), and for the other court sports discussed bespaced sufficiently for the size ball. Flared portions 306 are shaped asshown to minimize snagging on the tennis net (1-¾″ square openings) andthe center tie-down strap.

Collector blades 26 may be of a modular design, to increase or decreaseoverall length, or alternatively be extendible/retractable. Accordingly,when extending the blades, one or more additional ball collectionsensors may be installed on blade modules/sections for differentmulti-ball modes.

EXAMPLE 3

A prototype retriever 12 that incorporated elements illustrated in FIGS.9-11 and 18 (the blades and the ball tray with a single ball support andwithout active detents) was constructed on a Traxxas Model 83024-44TEC2.0 AWD chassis. The drive system was a Titan Model. No. 12T550brushed motor consisting of a high speed 12 turn design and integratedcooling fan, with an integrated cooling fan and a 3 mm output shaftpowering the drive train via a pinion gear and spur gear-driven shaftconnected to both front and rear axles to provide all-wheel drive,controlled by a servo control, and powered by a LiPo 5000 MAh battery(amp draw rating, at 7.2V=3.2 amps max). Ball retrievals were conductedfor one, two, and three balls at differing locations on a flat surfaceapproximating the size and configuration of one side of a tennis courtdimension (note—testing was conducted during the Covid-19 pandemiclockdown with tennis facilities closed). The tray and lower bladeclearance to ground was ⅛″-¼″ with the ball tray's leading lip havingabout a 1/16″-⅛″ ground clearance. The tray was fabricated from 1/16″thick aluminum sheet stock and integral with 3″ spaced apart walls formounting the blades to create a 3″ wide ball collection space. Ballswere successfully retrieved from several spaced-apart locations and theninertially ground-released toward a target via decelerating/stopping theretriever. It was observed that employing manual control throughout amulti-ball retrieval necessitated some training time and acquired skill,particularly to be able to control the speed of the retriever duringturns in order to avoid premature release of balls from the collectionspace between the blades prior to reaching the release point. Thisproblem readily resolved by (i) programming the retriever system to handoff manual control to automatic system control upon one or more ballproximity sensors sensing ball capture, and/or (ii) including one ormore pairs of active detent means (e.g. solenoid-actuated pistons orbuttons, a robotic claw, and so forth).

EXAMPLE 4

The retriever in Example 3 was run during singles match conditionsemploying a grid pattern as described above. Retrievals were conductedin the on-court and off-court areas designated by rows labelled as“net”, “1-3”, and “line” for the service court areas with “mirror image”rows as described for back court retrievals, and the off-court areasalso as described above. The voice commands further included commands to“slow”, to turn (“left” or “right”), and then “good” to stay on heading.Retrievals were also conducted on adjacent unused courts using thissystem. The system worked well in regard to the voice command structureand the operator estimating the balls' locations and making adjustmentson the ball approach using the appropriate adjustment commands. Aminimal number of retrievals were aborted.

Various ball tracking patents and systems directed to tennis matchball-tracking may be utilized in the invention, e.g. as described inU.S. Pat. No. 7,203,693 or as implemented in the HawkEye® system. Videostream ball-tracking data can be tapped into and accessed by system 10for accessing ball location and dynamics for retrieval purposes, whichdata may be accessed a priori to final [line call] processing. Thelatter may require a higher degree of precision (e.g. ball in-ball outline determinations) than that required to direct the retriever for aretrieval operation. Retriever 12 may include noise-reduction technologyto minimize the noise effects and potential for player distraction orinclude a [user-selectable] sound generator for generating a lessoffensive, masking sound. Orientation means 17 uponstartup/initialization locks in an initial court position orientation,preferably via an operator positioning the retriever alongside a netpost in its standby position, initializing that position as input intoprocessor-controller 21 that includes instructions which upon executionapply that standby position into ball retrieval computations so as toaccurately direct (and factor into tracking) the movements of retriever12 in commencing a retrieval and for guiding retriever 12 back into asuitable standby position and orientation.

An exemplary 3-axis MEMs magnetometer is the LIS3MDL model from ST(available from Pololu in a carrier with voltage regulator for 5 V powerapplications with an I²C clock and data line 5 V interfacingcapabilities (Model no. 2727)). The three magnetic field strengthreadings (as noted, it should suffice to use just two taps for a planarsurface application as here) are available through a digital interfacewhich can be configured to operate in either I²C (TWI) mode or SFI mode,although since retriever 12 will be positioned on and moving around aplanar court surface, the ‘x’ and ‘y’ outputs should suffice for mostpurposes as inputs into processor-controller 21.

The housing 12 should be selected from materials that do not interferewith the accuracy and operability of orientation means 17; whenemploying a magnetic field-based device such as a magnetometer, suitablehousing 12 and securing means 14 materials should be non-ferromagnetic,e.g. plastics or polymers.

Professional tournaments utilizing ball-persons by also utilizing theretriever can optimize the combination by decreasing the number ofrequired ball-persons. For example, tournaments often position twoball-persons at the back of each court side as well as at least one andusually two near the net, one by each net post. One or both ball-personsby the net posts can be replaced by one or more retrievers that canretrieve balls in or near those areas and then return the balls in thedirection of the ball-persons behind the baselines. For example, evenreplacing just one ball-person by the net posts decreases the staffingrequirements and facilitates that ball-person's job in retrieving netballs toward the other sideline. Of course, the alternating of servesfrom one court side and to the other more often than not results in“lets” (balls hitting the top of the net and dribbling over), leading toretrieval and match delays minimized by substituting the retriever fornet-caused or net-side retrievals. Accordingly, employing the retrievernot only speeds up play but also decreases the distraction to theplayers since the retriever is smaller and less visible (for example asthe players are reassuming their service and receiver statuses andball-persons are not in their direct or peripheral lines of view). Asdiscussed above, optional LEDs 29 (e.g. the model CX013 dot red linelaser diode module with adjustable laser shape, 3 v, 5 mw, 650 nm, mfr.by Changxing) on retriever 12 project ahead via visible light lines ofretriever 12's ball-collecting path or “swath” to indicate an accurateheading toward a ball to aid in manually-directed retrievals byfacilitating manual course-heading adjustments.

For a system 10 configured for 2-player use, processor-controller 21 caninclude a routine to track/allocate for each remote/player an allotment(like a chess match clock) to allocate their respective retrievalallowances. This encourages judicious deployment and encouragesconserving retriever power until later in a match when retrievals aremore critical. Status can be provided via a smart phone app. This canalso promote player safety, in that having the retriever respond to theside of the court called from necessitates that a player is alerted tothe retriever moving on that side of the court—a beeper and/or flashinglight on housing 15 can also promote safety. Accordingly, indicator 48or 65 as discussed above may comprise an audible alarm (e.g. a beeper)to alert nearby players to the movement of retriever 12. In order toenhance system safety reliability, a microphone 812 or other suitableacoustic sensor is preferably included to check the functioning of alarmindicator 48 or 65, similar to that used in the second generation GoogleNest Protect which offers a built-in microphone to self-check itsspeaker and horn by means of the speaker and horn emitting a short testsound, with the microphone monitoring the expected sound. In the case ofGoogle Nest, if the microphone cannot confirm operability, an indicatoris activated. The invention accordingly preferably includes this safetyprotocol and additionally is configured to output a stop oraborting-return to standby signal to cease the retrieve in progress. Theaudible warning system may be configured to activate selectively foroff-court retrievals or retrievals outside a programmed or user-selectedretrieval zone(s). These safety protocols of course can be interfacedand coordinated with the proximity detector 42 or 509 to further providefor court safety during retriever ops.

System 10 can further include fence-mounted sensors for mounting on thefences behind the players to alert a player to the presence of a ballrolling toward the player from behind and which can further include anaudible alarm. This relates to the safety aspects referred to that theinvention provides not just for balls in the playing area but alsobehind the player.

As noted above, processor-controller 21 includes a programmable moduleand microcontroller for receiving and processing input signals and forgenerating output signals for controlling retriever 12 and components.In addition to the input signals and output signals already noted,available (in some instances dependent on the particularembodiment/configuration of system 10) inputs include input signals fromretriever orientation sensor means 17, retriever location determiningmeans 19, sensors 32, 33, 42, and 509, T/R 43, commands from T/R 44(e.g. ball location coordinates), control signals from remote T/R 62,and T/R 69; processor-controller 21 outputs include but are not limitedto control signal outputs to retriever steering means 20, indicator 48,(active) detents 50, ball ejectors 52 (and solenoid 400), 420 and 512,and means 600 for rotating ball support 54. Processor-controller 21 isprogrammed to process the input signals and output the correspondingcontrol signals as is described herein and otherwise in accordance withprogramming known to one of ordinary skill in the art.

It should therefore be understood that the description and embodimentsdescribed and claimed herein should not be limited thereto. Accordingly,the invention may include additional variations within the knowledge andcapability of one of ordinary skill in the relevant art. For example,alternatives to the solenoid-piston type impeller ball-strikingmechanisms described above include other impeller-drive devices, such asin US Serial No. 20110303208A1, published Dec. 15, 2011 (G. Taylor) andincorporated herein by reference, where the impeller assembly mayinclude (a) a gear-driven rotating member employing a coupling mechanismthat includes reliant biasing means mechanically attached to theimpeller assembly, or (b) a rack and pinion system with the pinionrotationally coupled to the impeller assembly. Given the desire toprovide a compact or low-profile retriever in the present invention,solenoidal mechanisms generally provide the better alternative.

As mentioned above, in order to optimize retriever 12's performance timecharacteristics, it may be desirable to utilize multiple retrievers,e.g. two retrievers per court, one dedicated to each court side andpositioned accordingly, in the direction of its designated side atstandby. Alternatively, retriever 12 has a turret rotatably mounted onchassis 14, thereby allowing retriever 12 to move toward either courtside quickly without rotating chassis 14, and where the turret can berotated into position even as retriever 12 is moving or starting to movein the direction of the retrieval, optimizing retrieval time. US PatentApplication No. 20100155156A1, R. Finkelstein, Jun. 24, 2010, and USPatent Application No. 20170153091A1, K. Diaz, Jun. 1, 2017, bothincorporated herein by reference, describe top-mounted turrets that withminor modifications are within the scope of the invention.

Also as discussed, means 19 for determining the location of retriever 12includes GPS-enabled receivers, and other means, e.g. wheel encoderspositioned on one or more wheels with input(s) to processor-controller21. For the latter, processor-controller 21 includes locationcalculation programming with instructions that upon execution compute apresent location of retriever 12, as input into otherprocessor-programmed modules such as the steering and docking controlroutines. Duplicate location determination means may be included shouldredundancy be desired for comparison or backup purposes, with anoptional automatic or manual heading direction override.

Different court surfaces may dictate different functionalities andtherefore different modifications to and versions of retriever 12.Well-groomed clay or HarTru®-type soft court surfaces may necessitate orsuggest the use of wheel configurations having better traction whileleaving minimal tracking than others—narrow wheels generally providebetter traction but leave deeper tracks, therefore aft-mounted surfacebrushes (e.g. one behind each rear wheel) can smooth the tracks butincrease drag. The retriever should preferably go forward and nottraverse a path in reverse, which affects the path selection and itsprogramming. Wider wheels or treads would slip or skid more, but leavewider, less pronounced tracks. Retractable brushes are advantageous toreduce drag, but add more complexity, and are practical only inconjunction with the requisite path programming to maximize thedeployment effectiveness. Grass courts tend to “brake” or slow down arolling ball more than other surfaces; also, the ball does not roll aswell upon collection and encumbers the retriever, so full ground returnsare not as desirable as securing a collected ball just above the grasscourt surface (e.g. by means of slightly raising lip 412 upon ballcollection onto lip ball surface 413) until reaching the ball releaselocation. Although a standard ground release may then suffice, employingan ejector 420 provides a higher release velocity which is desirable ongrass.

Multi-ball collection can utilize lip 412 because collecting two or moreballs and rolling them in contact with one another can lead to“turbulence” in the balls' rolling dynamics from the surface-to-surfacefrictional contact; hence, each ball's surface rolling can be impactedby the dynamics introduced into the system. Lip 412 helps to minimizethis phenomenon by providing separation between the collected balls andelevating one or more balls above the court surface, eliminating therolling dynamic and facilitating “non-turbulent” retrieval dynamics.

Global Navigation Satellite Systems (GNSS) enhancements to positionaccuracy include “dual-frequency” GNSS support. Dual-frequency GNSSmeans that the receiver tracks more than one radio signal from eachsatellite on different frequencies—for GPS, this is L1 and L5, and onGalileo E1 and E5a. Utilizing two GPS frequencies versus just oneimproves location determination accuracy (user range error (“URE”)) byorders of magnitude (by correcting signal distortions caused by Earth'satmosphere), up to in some systems even a few centimeters locationdetermination accuracy. Costs should come down and make this affordableeven in the basic versions of the invention. Systems that enhancepositional accuracy inter alia include DGPS (Differential GPS), AGPS(Assisted GPS), RTK (Real-Time Kinematic), and e-Dif (extendedDifferential). DGPS utilizes a base or reference station GPS receiverand a second roving GPS receiver, to improve accuracy to within a fewmeters or better. Advanced L5/E5a signals are less prone to multipatherrors and therefore can be used to refine position accuracy to theorder of 30 cm. Accordingly, dual-frequency GPS T/R's 43 and/or 44 (thatis, one situated on retriever 12 and the other off-board, e.g. on thedocking station 64 or the remote 60 utilizing a combination of workableprotocols, handshakes and the like) can deliver the requisite accuracyover present one-frequency systems. It is also recognized thatpredictable and customary advances in the arts such as advances in GPSaccuracy can contribute to system performance.

Retriever 12 can further include a towel holder or dispenser or the like(not illustrated) to enable a player to towel off as is provided byball-persons at tournaments. A suitable protocol for enabling this as anoption is having a remote control keying sequence or employing a voicecommand, such as “Towel” where the ball(s) are returned and thenretriever 12 is queued to approach the player to utilize this feature.

It is again noted that the invention as described herein should beinterpreted as not limited to the specific embodiments described but toinclude variations recognized as equivalents in the art(s). Theinvention thus includes in-process developments such as highly sensitiveacoustic pickup devices (as discussed above) positioned on the retrieveror docking station (e.g. on one or more of retriever 12 docking station64s' T/R devices 43, 44, and 69) including sub-vocalization capabilitiesfor controlling retriever 12 as discussed. The invention may also finduse in other sports such as baseball that similarly employ ball-kids toretrieve loose balls. A player was injured recently when, whileattempting to catch a ball in foul territory, he fell through an openingin the screening used for ball-kid access to the field. One or moreretrievers 12 may be positioned at standby within a smaller field accessopening, e.g. one built into an advertising panel inset into a wallalongside the stands, and controlled remotely as described herein,providing a safer ball retrieval system. System 10 can then furtherconstitute a comprehensive ball handling and management system forcollecting stray baseballs and depositing same into a receptacleadjacent to the docking station following each retrieval, optionallywith sanitizing means such as an UV apparatus described herein. Thisalso minimizes human contact with the ball, desirable in maintaining asanitized environment, especially important during an epidemic such asthe SARS-Covid-2 epidemic. Of course, ball-kids interact with baseballgame attendees, often tossing them the ball as a game souvenir. The ballhandling system can offer an alternative, e.g. by identifying anattendee in proximity to the retrieval and awarding the person that oranother game ball (perhaps one autographed by the player who struck theball).

What is claimed as new and desired to be protected by Letters Patent ofthe United States is:
 1. A tennis ball retriever for retrieving a tennisball on a tennis court and for controllably releasing the ball during amatch between at least two players, said retriever having a fore end andan aft end, comprising a chassis; wheels mounted on said chassis; adrive system coupled to at least one wheel; a means for steering theretriever; a receiver for receiving retriever control signals; aretriever processor-controller, having inputs for receiving control anddata input signals and outputs for retriever command and controlsignals, said processor-controller being programmed with instructionsthat upon execution generate output signals based on the input signalsfor controlling the retriever; an on-board power supply; and a pair ofopposing blades positioned on the retriever fore end and spaced apart todefine a fore ball collection space therebetween having a widthsufficient to receive and hold a collected ball until release; and aball tray positioned between or integral with the bottom portions of theblades and extending alongside therewith, and at a height sufficient toprovide ground clearance without impeding the rolling collection of aball thereon, and said ball tray including: a plurality of spaced-apartball support surfaces including an aftmost ball support surface and aforemost ball support surface, and a wall aft of the aftmost ballsupport surface serving as a passive detent for securing a ball on theaftmost ball support surface; a plurality of spaced-apart ball proximitysensors positioned alongside said ball collection space to cooperativelysense the presence of a ball on each of said ball support surfaces; aplurality of spaced-apart active detents positioned within thecollection space so as to trap a collected ball in cooperation with theball proximity sensors on a ball support surface; and a means forgripping and securing a ball positioned at the fore end of theretriever.
 2. The retriever of claim 1, wherein each said sensor ispositioned along a blade sidewall, within or under a ball supportsurface comprising a through hole, or a combination thereof.
 3. Thetennis ball retriever of claim 2, further comprising outwardly-flaredtips for deflecting a ball into the ball collection space.
 4. The tennisball retriever of claim 1, wherein at least one of the active detents isa solenoid-operated piston member.
 5. The tennis ball retriever of claim4, further comprising a ball ejector positioned aft of the aftmost ballsupport surface for ejecting a ball therefrom in a horizontal direction.6. The retriever of claim 6, further comprising a housing mounted onsaid chassis for securing retriever components thereon.
 7. The tennisball retriever of claim 1, further comprising an aft ball collectionspace and a first ball ejector positioned therein for releasing acollected ball in a vertical direction.
 8. The tennis ball retriever ofclaim 1, further comprising an orientation sensor.
 9. The tennis ballretriever of claim 9, wherein the processor-controller is furtherprogrammed with instructions that upon execution and based on theretriever ball release location determination compute a retriever returnpath and output a corresponding control signal directing the retrieverto move to an established off-court standby location following a ballrelease.
 10. A tennis ball retrieval system for retrieving andcontrollably releasing a tennis ball on a tennis court and forcontrollably releasing the ball during a match between at least twoplayers, comprising: (i) a retriever having a fore end and an aft end,comprising a chassis; wheels mounted on said chassis; a drive systemcoupled to at least one wheel; a means for steering the retriever; areceiver for receiving retriever control signals; a retrieverprocessor-controller, having inputs for receiving control and data inputsignals and outputs for retriever command and control signals, saidprocessor-controller being programmed with instructions that uponexecution generate output signals based on the input signals forcontrolling the retriever; an on-board power supply; a pair of opposingblades positioned on the retriever fore end and spaced apart to define afore ball collection space therebetween having a width sufficient toreceive and hold a collected ball until release; at least one detent onat least one said blade for retaining the ball in the collection spaceduring movements of the retriever prior to a controlled release of theball toward a target; a means for gripping and securing a ballpositioned at the fore end of the retriever. and an orientation sensor;(ii) a remote control for signaling to and controlling the retrieve;(iii) a docking station for docking the retriever at the standbylocation; and (iv) a means for attaching and securing the dockingstation to a net post.
 11. The tennis ball retrieval system of claim 10,wherein the remote control includes a voice command capability fordirecting retriever actions.
 12. The tennis ball retrieval system ofclaim 10, further comprising a wheeled frame with the retrieversecurable thereon and having the means for attaching and securing theframe to a net post mounted thereon.
 13. The tennis ball retrievalsystem of claim 10, further comprising (i) a ball locator and trackerfor locating a ball and providing an output data stream identifying theball location, and (ii) a transmitter for transmitting the ball locationdata stream to the receiver, and wherein the processor-controllerincludes programming instructions that upon execution direct theretriever to the ball location.
 14. The tennis ball retrieval system ofclaim 13, wherein the ball tracker and locator and transmitter areattachable to a net post.
 15. The tennis ball retrieval system of claim10, further comprising a ball locator and tracker positioned on theretriever for locating a ball and providing an output data streamidentifying the ball location to an input of the processor-controller,and wherein the processor-controller includes programming instructionsthat upon execution direct the retriever to the ball location.
 16. Thetennis ball retrieval system of claim 10, further comprising a ballejector positioned aft of the aftmost ball support surface for ejectinga ball therefrom in a horizontal direction.
 17. The tennis ballretrieval system of claim 10, further comprising a retriever statusindicator light.
 18. A method for retrieving a tennis ball during amatch between two or more players, comprising: providing a retriever forcollecting a ball on a sports playing surface and for controllablyreleasing the ball, said retriever having a fore end and an aft end,comprising a chassis; wheels mounted on said chassis; a drive systemcoupled to at least one wheel; a means for steering the retriever; areceiver for receiving retriever control signals; a means fordetermining a location of the retriever; a retrieverprocessor-controller, having inputs for receiving control and data inputsignals and outputs for retriever command and control signals, saidprocessor-controller being programmed with instructions that uponexecution generate output signals based on the input signals forcontrolling the retriever, and the processor-controller is programmedwith instructions that upon execution process the retriever locationinput signal into a retriever control output signal for directing theretriever to move to a programmed ball release location and release theball in accordance with the processed programmed instructions in acalculated direction and with a calculated velocity toward a targetlocation; an on-board power supply; and a pair of opposing bladespositioned on the retriever fore end and spaced apart to define a foreball collection space therebetween having a width sufficient to receiveand hold a collected ball until release; means for gripping and securinga ball positioned at the fore end of the retriever; and at least onedetent on at least one said blade for retaining the ball in thecollection space during movements of the retriever prior to a controlledrelease of the ball toward a target; and (ii) a remote control;positioning the retriever in a standby location; directing the retrievervia the remote control to collect a ball; directing the retriever to acomputed release point pursuant to the processor-controller programmedinstructions; and releasing the ball from the retriever at a computedspeed and direction toward a target location.
 19. The method of claim18, wherein the remote control includes a voice command capability fordirecting retriever actions.
 20. The method of claim 19, wherein thevoice command includes a grid protocol and commands for directing theretriever to steer toward a ball during approach to the ball.